Mastodon does that (check for the AS2 context url), so I suppose that would mean a large majority of Fediverse servers do what they absolutely shouldn’t be doing. However, I don’t blame them. The AP/AS2 guidance is confusing, at best, in my opinion. In fact, I’ve seen people like @evan explicitly suggest that approach (using context URLs as an extension mechanism – even though Mastodon embeda their own context for several technical reasons).
My current understanding is that JSON-LD was part of Snell’s Activity Streams vNext prior to the W3C WG and that it served some useful political purposes at the W3C early in the process (before the LD/RDF supporters left to form the SoLiD project). At this point, I’d like to see it dropped for plain JSON consumers, with an option for Linked Data interop (ActivityPub-LD). I think JSON Schema could be a benefit to both use cases (in the absence of, or in addition to, a normative ontology for the JSON-LD/RDF users). Messages could include a $schema property for JSON Schemas and also include a @contextif they want to interop with JSON-LD/RDF.
The biggest problem I see is that at least one of the AS2 Rec authors is still strongly pushing JSON-LD as an extensibility solution for non-LD consumers in spite of the obvious issues and extensive feedback from AP developers.
To be clear, I do think that JSON-LD supports extensibility, but only if you’re using JSON-LD processing.
You are assuming that “fields are defined in the spec” seems to be an adequate substitute to “fields are defined in the document”. To this I say: which spec? I assume you mean the AS2 and AP specs, but those aren’t the only specs in existence. And that’s the crux of the whole issue – it should be possible to implement multiple specs. It should be possible to implement conflicting specs. You shouldn’t need to coordinate between specs or systems in order to resolve those conflicts.
Even within the AS2/AP specs alone, there are still properties that can have 1 or many values. This has nothing to do with JSON-LD; JSON-LD (expanded form specifically) only makes this reality more visible, by forcing you to put square brackets around things instead of coercing them away. Similarly, JSON-LD (expanded form particularly) forces you to be explicit about which nodes are @id nodes and which nodes are @value nodes. If you don’t want to be forced to do this, then someone else can do it for you by coming up with a @context that you can inject as just one more key that you can ignore. The difference between “AS2 (plain JSON)” and “AS2 (compacted JSON-LD)” is purely in the one @context declaration. Which, again, you can ignore if all you care about is the AS2/AP specs. It only matters once extensions start to matter for you. But you still have to deal with one-or-many to, one-or-many attributedTo, one-or-many object, and so on. These are all things that are in the spec right now, that have nothing to do with JSON-LD.
Obviously we disagree here, but it’s striking to me because the idea of a Web browser is clearly something that pretty much everyone already understands and uses all the time! You and I are both using one right now! I don’t think the Web is just an interesting tech demo – I think it’s the largest publishing platform in existence. Is that suddenly a “quirky technical thing”? Is the natural order of things to publish only to specific services, perhaps interconnected perhaps not, and to start over from scratch losing your social network every single time you switch services or make an account on a new one?
Maybe I’m missing something, but it seems very straightforward to say that ActivityPub objects are in most cases Web resources – they have an HTTPS URI and you can do an HTTP GET to fetch them. It seems straightforward to say that if these are Web resources, then it would be valuable to have a way to manage them via a Web server. One way of doing that is to use HTTP verbs like GET, POST, PUT, PATCH, DELETE. This is what LDP does, loosely speaking. The ActivityPub way is to use specified Activity types like Create/Update/Delete for CRUD stuff, and to bolt on delivery of those activities to specified audiences, as a notification that the activity took place, triggered by a POST to your outbox. It builds off of LDP’s “inbox” and the generic LDN (Linked Data Notifications) mechanism, but with the added restriction that the notification resource is specifically an AS2 document where the root object is an Activity or subtype.
Whatever the mechanism, everything is a Web resource. To say that this is “not feasible” or “hampering the fediverse” is entirely the wrong framing to me; rather, we should not hamper the Web. The Web as a whole is far, far, FAR bigger than the fediverse. Having standardization around its social components is a worthwhile goal. I don’t think anyone should limit their imagination purely to cloning Twitter or cloning Reddit or cloning Facebook. Having a bunch of implementations that each cover a very narrow use case, where you can’t have consistent identity across them all, you can’t have consistent followers across them all, you can’t have a consistent way of managing what you post across them all? That’s the biggest fundamental flaw in the fediverse today. Nevertheless, existing services have the option to integrate by transforming from their internal format into some response that’s more useful. This should not happen over proprietary vendor-specific APIs. This should happen in a standardized way.
This is the part where I ask, what does it mean to “present an alternative”? Is the goal to replicate corporate centralized social media design, but “open” and “decentralized”? I think this is thinking too small. To use an analogy from environmentalism, what do you accomplish by shifting everyone from industrial farming to free-range cattle? If the total consumption of beef remains the same, then the total production of cows must likewise remain the same. And the CO2 production of cows is significant. So it follows that in order to reduce CO2 output from cows, then it is necessary to reduce the production of cows, and in order to reduce the production of cows, then it is necessary to reduce the consumption of beef. This is assuming you care about CO2 production. In social media terms, “CO2 production” is a stand-in for negative outcomes like context collapse, reliance on services, and so on.
What the Web allows people to do, ideally, is to express themselves, to publish what they want, where they want, how they want. This is what I think we should encourage. This is what I think we should facilitate. You shouldn’t have to post your short thoughts to Twitter, and your pictures to Instagram, and your videos to YouTube, and your longform articles to Medium, and so on. Subbing in Mastodon for Twitter, or Pixelfed for Instagram, or Peertube for YouTube, or WordPress for Medium, all these things are technically an improvement in that you are no longer using the “corporate centralized” offerings that do not respect you as a human being, that place profits above your communications. But you’re still not able to express yourself fully or without limitations. You are still bound by the limitations of Mastodon, or Pixelfed, or Peertube, or whatever you’re using. Namely, you have to make an account on each one of these, and you can only see what they allow you to see, in the way that they allow you to see it. This can’t be the end goal. We ought to empower people and facilitate their communication in a way that doesn’t require them to make multiple accounts or be bound to each of those domains.
It would help to, again, be clear about what exactly those user needs are, and what the goal of the system is.
It would also help to recognize that “implementers are reluctant to use JSON-LD” is not painting the full picture. The reality is that implementers are not even following the spec. In many cases, they are following what Mastodon does, or what Lemmy does. This is a very fragile way to build out a network. It requires you to do n-n communications with every single project that you’d like to be “compatible” with. Reading twenty different FEDERATION.md documents only gets you so far. It’s less about the “suitability” of JSON-LD for any given purpose, because the problem is actually occurring at a far lower level. If I had to point to any particular reason, it’s not that cognitive cost is too high or that benefits don’t justify complexity, it’s that the “federated protocol” is more defined by what the de facto leader Mastodon does than it is by the spec. Because Mastodon has the “users”. Because people want their “posts” to show up to those “users”. The same thing happened when Reddit made unpopular changes and people flocked to Lemmy as one of the only alternative options. Newer projects that sprung up didn’t target compliance with ActivityPub, they targeted compatibility with Lemmy. This is, again, before we even get to the point where evaluating the pros and cons of JSON-LD matters at all.
I apologize if it came off as arrogant or elitist, because that’s not my intention. My intention is not to bemoan the low adoption of “proper” JSON-LD, but simply to demonstrate that it serves a need. That need being “extensibility”. There are other ways to extend AS2, but JSON-LD is the one that doesn’t depend on you submitting to the AS2 context document as the One True Spec. The other way that does require this is the “extensions policy” that adds properties into the AS2 context document, as mentioned earlier.
Aside from that, you can do extensibility with JSON-LD and without JSON-LD processing, by more or less reinventing the JSON-LD expansion algorithm. Which probably doesn’t count, huh.
For the brave of heart, the full JSON-LD expansion algorithm as described in JSON-LD 1.1 Processing Algorithms and API is 20 steps long with several sub-steps, but you could probably get away with implementing a much more limited subset of it. A lot of steps deal with keywords that aren’t relevant to AS2. You’d probably be able to get away with implementing just the “IRI expansion” JSON-LD 1.1 Processing Algorithms and API called out in AS2-Core, and possibly “Value expansion” JSON-LD 1.1 Processing Algorithms and API if the value of the property is compacted.
Such conflicts are rare (haven’t happened yet in Fediverse, as far as I know) and can be resolved by social means. If conflict is unresolvable, developers may decide to use a fully qualified property name (an IRI), but we don’t have to call that “JSON-LD”. It could be “JSON with a fancy property name”.
JSON-LD is unpopular, and I think its use should be reconsidered. I wonder how many people decided to not implement ActivityPub because they didn’t want to deal with JSON-LD. How many wasted their precious time reading about JSON-LD and implementing it, only to learn that is it not necessary to federate with other software (and in fact may harm interoperability). How many hours were spent arguing about all of that…
I think the next version of the spec should make it clear that ActivityPub objects are plain JSON.
I dunno, this makes it sound incredibly difficult to achieve any kind of standardization. I mean the point is to be able to communicate right? I get that we’re going for a decentralized protocol but the way I see it the protocol definition itself cannot be truly decentralized - otherwise how do we agree on anything? Standards are in that sense inherently centralized.
And again, I don’t think that there would really be a lot of those conflicts in practice just because we eschew JSON-LD in favor of plain JSON. I don’t think JSON-LD is necessary to avoid conflicts.
HTML/CSS/JS (or WASM if you prefer) were built to support arbitrary content with arbitrary behavior and arbitrary UI and presentation. That’s why web browsers are insanely complicated, so complicated that implementing new ones takes years if not a whole decade to implement, and even then you likely don’t support everything. Maybe I am misunderstanding your vision here but there is no way that ActivityPub becomes some kind of browser of objects on top of the web with the same level of complexity and possibilities. That’s simply not feasible if you ask me.
I don’t really see the comparison with a web browser - ActivityPub objects do not specify any presentation or style, and I don’t think we want them to have that.
I understand this vision and I see how it maybe could happen. But I do want to question a bit: Why can’t that be the end goal? If that is what people want, then it makes sense. As someone that mainly prefers the reddit style of social media, I’m not exactly interested in a platform trying to be YouTube, Instagram, WordPress and Reddit all at once. People prefer different kinds of platforms. It kinda makes sense that people would gravitate towards platforms that cater to their preferred style of social media.
I mean isn’t it telling that the biggest successful fediverse projects are those that (more or less) clone corporate social media? You could take a step back and question why corporate social media in the way that it exists now is popular: Because people want it like that.
Said in another way, if people actually wanted this omega-platform you propose with YouTube, Medium, Reddit, Instagram and Twitter all bundled up into one thing and capable of presenting all that content in a coherent way, why isn’t there a corporate-backed popular social media platform that works like that? Maybe the answer is simply that people are not interested in such a platform. Why use a swiss army knife when I prefer a dedicated can opener?
I know we’ve kinda strayed into the weeds here, but I agree with this. Those flaws sound like they should be solved. I just don’t think JSON-LD is necessary to achieve that and actually I would even say it hampers solving it, due to its complexities.
My point is that, the fact that implementers aren’t following the spec and instead trying to be compatible with each other in an ad-hoc fashion should be a signal that the spec is too complicated or difficult or somehow unsuitable to actually implement. I don’t think the solution to that is to say “we just need implementers to actually implement the spec, then we would be good.” That’s not a solution to the problem, it just ignores the problem. If JSON-LD actually served a need, implementers would probably use it - but they don’t.
How do we get implementers to actually follow the spec? I think the only way is to simplify the spec. Make it easier and less daunting to implement, and implementers will happily follow suite. Removing JSON-LD is a major part of that, I would say. Certainly not the only part, but a major one.
I resonate so hard with this . I literally joined this forum as I started an ActivityPub side project and started reading about and learning ActivityPub. It’s obviously very early and I don’t know where it will lead but I am actually trying to implement ActivityPub. The more I implemented and the more I read in the spec the more frustrated I got at how complicated it was. Especially JSON-LD was frustrating as again, I had never heard about it before and it just seems like a no-brainer that a federated, widely used standard such as ActivityPub ought to use the lingua franca that is JSON, and not some weird dialect of it (i.e. JSON-LD). And yet it does use JSON-LD. Sigh.
And because of that, instead of actually implementing my side project, I am now sitting here and moaning about how ActivityPub is complicated and JSON-LD should be thrown in the trash but wait we can’t do that because it would break backwards compatibility completely. And that just makes me even more frustrated, as what I see as mistakes don’t seem to have a way of being corrected.
I even spoke to another developer who also tried implementing some ActivityPub libraries a few years back and they basically agreed with me that it is complicated and they said they gave up on their project because it burned them out.
So yea, your comment really made me think about the time I spend here writing instead of writing on my side project…
Again, this only works when the fediverse remains small, and “fediverse” is all you want to do. When you widen your scope to “World Wide Web”, you have to deal with a much larger, open-world system where “social means” are not always available to you. I’m familiar with the YAGNI argument, but I don’t think it holds beyond a certain point where you actually do start needing more complexity.
I posit that there’s not really any functional difference. Given that your current document is already “plain JSON AS2”, and you want to add the following property to your document:
then how is this any different? Is the argument here that compact URI expansion SHOULD NOT be used, and that IRIs should always be used, and that POST to outbox/inbox SHOULD NOT inject any additional contexts other than the AS2 one? Because that seems to be the way to accomplish what you’re asking for based on what I understand. Really, all the additional context declaration does is let you do something like this:
and if you want to do away with that, then sure, go for it. FEP-e229 has “Consider not using prefixes or term definitions” as one of the recommendations for LD-aware producers for a reason. But it seems like something that requires careful consideration before mandating it for everyone.
Is it possible that you’re focusing too much on JSON-LD as a source of complexity when it might make zero practical difference for your use-case? To me, the complexity in the fediverse is not in JSON-LD. It’s in all the other parts! AS2 lets you ignore JSON-LD entirely if you’re willing to also ignore extensions entirely. At that point, what issue does JSON-LD pose to your implementation?
Say you implement AS2 as plain JSON, and say you implement the ActivityPub spec. The problem with the fediverse is that this is not enough. If your goal is to talk to Mastodon or talk to Lemmy, you now have to go do a bunch of nonstandard idiosyncratic stuff and follow some other specs or spec profiles before you can even send a “Hello, World” post. Again, JSON-LD has nothing to do with this.
For practical purposes, everyone can just generally ignore the JSON-LD @context and implicitly share the same context as Mastodon – this means that when they say discoverable, you have to know ahead-of-time what it means exactly… but if someone says toot:discoverable then you have no idea what it means because you’re ignoring JSON-LD. Well, congratulations, you’ve reduced complexity at the cost of introducing fragility and limiting flexibility. Maybe that’s a tradeoff you’re okay with making. Maybe you’re fine with delegating your worldview to Mastodon. But once you make that tradeoff… why worry about JSON-LD anymore? It shouldn’t be frustrating you because you no longer have to deal with it. You’ve already accepted a “plain JSON” world where your context is implicitly defined ahead-of-time by whatever specs (official or de facto) that you chose to implement. I don’t understand why JSON-LD is an obstacle to what you want to accomplish.
Let me put it another way. Should that standard be available only as a human-readable document, where the requirements are not always clearly defined, and where the human reading it needs to understand it in depth before any single part of it becomes useful?
Or should that standard be available in a well-defined, concisely-bound way, such that it can be both human-readable and machine-readable, where the requirements are strictly described by a schema or ontology that tells you exactly what each property’s domain/range/label/comment/maxCardinality/etc are, and where you can easily use just one single property out of the standard, if that’s all you need?
We “agree” on something when the URIs match, or represent equivalent concepts (owl:equivalentClass, owl:equivalentProperty, owl:sameAs)
The “protocol definition” need only define side effects. Otherwise, the facts speak for themselves (literally! each RDF statement can also be referred to as a “fact”)
ActivityPub doesn’t have to be that “Web browser”. Actually, ActivityPub as a spec or protocol can be very simple! You have an inbox and an outbox, you POST activities to them, basically the server is just a dumb pipe that does nothing but store-and-forward the activities it receives, while processing the minimal side effects defined in the ActivityPub specification.
But if you want to talk to the “fediverse”, then ActivityPub is not enough. Right now, to engage with the fediverse, you need not only ActivityPub, but also HTTP Signatures (Cavage draft), WebFinger (with ability to resolve acct:preferredUsername@host for every single actor), and some additional restrictions depending on the project – for example, Mastodon requires your preferredUsername to be of a certain format and certain length, it requires your payloads to be generally of the Create Note shape, and so on. You end up having to read the documentation provided by the Mastodon project, and if you want to target compatibility with some other project like Lemmy, you’ll have to read their documentation too, and so on and so forth – this scales linearly with every single new implementation you want to support, including all of its quirks and idiosyncracies. Ad-hoc compatibility. It’s unsustainable. That’s what leads to burnout IMO, not JSON-LD (which can be ignored).
Now, consider the alternative model of the Web: you use ActivityPub to manage Web resources and send them around to other actors. Those ActivityPub objects are then handled in certain ways by whatever clients are plugged into each actor. You could have as many AP clients as you wanted to, and they all have the ability to put things into your outbox for delivery, or pull things out of your inbox for whatever processing they want to do. You could:
use ActivityPub to communicate like any other messaging platform
use ActivityPub to generate a website from HTML representations of objects
use ActivityPub to interact with remote systems by sending them commands
use ActivityPub to notify subscribers of new content
and much, much more. Mastodon could be a client. Pixelfed could be a client. Lemmy could be a client. They would each operate as mini-Web-browsers that also let you POST activities of a certain type to your social profile. I mean, they already do mostly act as mini-Web-browsers. They just don’t let you use the same profile – you have to create a new profile for each service. This is unnecessary, and AP C2S is the mechanism that would let them use the same profile to post activities. In much the same way you can use an email client on your desktop and in your browser and on your phone… some email clients use IMAP to let you take notes… some email clients are skinned to look like chat clients (DeltaChat)… the messages that are carried over the email protocol let you interact with remote systems, like replying to a comment on GitHub without actually having to visit GitHub… the possibilities are literally endless.
For simplicity, though, there are two pieces to the puzzle that I think are necessary for a Web-first approach
a generic browser, to consume AS2 objects. (browser.pub is really cool and I think it will continue to get better.)
a publishing tool or CMS, to produce AS2 objects.
Both of these would plug into ActivityPub to enable you to access your social network. The browser could authorize against your actor and browse your inbox. The CMS obviously could authorize against your actor and post to your outbox.
If you wanted to widen your scope to deal with not just AS2 objects, then that’s also possible.
There’s no problem if you just want to use one client that behaves like Reddit. That’s fine. But there doesn’t need to be an artificial split between different types of content or different accounts that represent the same identity.
I don’t think it’s because “people want it like that”. People go where other people are. Social media platforms are designed by different people, with different incentives.
I’d actually say that the “biggest successful” projects were just in the right place at the right time because people were leaving other platforms and were looking for the next closest thing. But that doesn’t mean we need to be limited to that paradigm forever.
The “omega-platform” already exists, and it’s called the Web. People already publish all kinds of resources of all different types, all the time. It’s just that it could be easier to publish to it. Right now, publishing to the Web can happen one of two ways:
On your own terms. You have a website or domain or some other presence that lets you host Web resources, and you publish them there.
On someone else’s terms. This is the “platform” model, where your SNS more or less manages everything for you, and you don’t get to control exactly how your content is published or in which forms. You just make “posts” which are owned by and distributed by the platform, not by you.
It’s not. It just makes it easier to build up your data piecemeal instead of wholesale, using multiple potentially-conflicting vocabularies without having to ask anyone for permission or to resolve conflicts. An open-world system. But for the exact flaws cited here, AP C2S is far more important and far more relevant.
Again, which spec? Hopefully I’ve already established that ActivityPub as a spec is actually quite simple… in fact, it’s too simple. It’s like trying to build a Web browser following only the HTTP spec, while leaving out HTML/CSS/JS entirely.
Throwing out JSON-LD doesn’t really do anything except hurt people wanting to treat AS2 documents unambiguously without being aware of AS2 ahead-of-time.
Speaking a little bit cynically here: the “need” that ActivityPub served is that it allowed Mastodon to do followers-only posts in a way that OStatus using Atom feeds could not (at least, not without convoluted schemes like giving every single follower an individually tokenized copy of your post feed). Mastodon didn’t need C2S and didn’t need JSON-LD and didn’t need a whole bunch of other things, because it was never trying to (fully) let its users publish Web resources. Arguably it used to be better at that, with plain HTML + Microformats being served at status permalinks, which it doesn’t do anymore. So from the beginning, it’s not like most people were aiming to implement ActivityPub. They were aiming to talk to Mastodon, which spoke its own dialect, which became a de facto standard because of everyone wanting to talk to Mastodon. If someone wants to formalize an entire networking protocol based off of what Mastodon does, then I guess they’re free to do that. Heck, I’ve thought about it before. But I don’t feel like putting a ton of effort into that when I’d rather put that effort into something more useful to me and hopefully generic enough to be useful in a context outside of just cloning Twitter.
Is wanting to use or develop an extension not sufficiently “advanced”? Because I think we generally agree here, it’s just the specifics where we might differ a little bit.
If I’m developing my first ActivityPub application I should be allowed to use any property or type without thinking about contexts and such. “Advanced” stage begins when developer wants to standardize their extension (by writing a FEP, for example).
…But at that stage, name conflicts usually can be resolved by social means.
So hear me out here: I think on one level everyone here is talking about whether the spec is fundamentally JSON-LD with a JSON-only variant and whether the inverse is possible. The crux of understanding status quo (and the biggest challenge with coming up with a good replacement, i.e. actually writing the JSON-only v2 or even v1.1) is how extension logic works in the “open world” of actual decentralization. I won’t belabor or summarize, but I really think this is the tricky bit. We can disagree about conclusions but it seems a common understanding that this is a philosophical disagreement about how best to experiment at the level of protocol extension without imposing priorities or constraints on that extension OR requiring central authority over tomorrow’s protocol. That’s the trillion-dollar question! Say what you will about blockchains, governance over those protocols once they’ve become enmeshed in chaotic international unregulable capitalism proves it is really really hard to govern a living protocol once real-world value is being transacted over it. If you don’t believe me, you should look at the size of Threads’ budget.
I think both sides here are very smart, and that needs to be said out loud due to a quirk I like to call the “engineer nerdswiping gene” which is an unfortunate side effect of being smart enough to have read this far in this hellthread. there are good and valid ways of iterating the AP spec that are fundamentally “JSON-LD spec with JSON-only implementation guide” (status quo, but as @stevebate points out, harder to finesse and probably require a much more detailed explanation of expanded form JSON-LD as an option for protocol extensions). There are also good and valid ways (probably many more of them!) for iterating the AP spec that flip the “cardinality” and make the spec easier to extend, but harder for JSON-LD implementations to interact with extending JSON-only implementations.
More importantly, this might be a distinction without a difference, because they require a lot of the same gymnastics that steve is pointing out we need either way-- if JSON-only implementations can originate extensions without accomodating JSON-LD in one form or another, we’re just doing the same amount of hack jobs, except on the protocol upgrade side of things (i.e., JSON-LD implementations’ interop is broken until each major version…). Pain upfront or pain later, in many ways-- like so many engineering tradeoffs, the question is who does the hard work when, there’s the same amount of hard work either way.
So the common ground might actually be more useful to explore than the holy war. I will also add that the DID and VC working groups grappled with all the same problems for the last 5 years; the hardline “JSON-only” folks ended up going to IETF and restarting the standardization process in many ways, which fundamentally makes the either/or engineering tradeoffs a matter of zero-sum, wontfix non-interoperability. The resolution of the interop problems will come when either the JSON-only-first or the JSON-LD-first verifiable credential standard becomes betamax and fades into history, it seems. That feels like a governance failure mode, because one side or the other will have wasted a decade in that future. Who said capitalism was efficient!?
Anyways, that’s about all I can say about “the holy war” I’ve referred to in other threads. That’s one layer on which we’re disagreeing which is literally ideological-- how do we think standards should be written, which is the fundamentally saner approach to these very hard problems.
On another totally distinct level, let’s call it developer-experience pragmatism, there is a very real developer burnout issue here. There are like 200 partial implementations of AP that were never finished because no one started out wanting to deal with all this. As strypey mentioned in another thread, this is partly because making “federated software” has all the problems of making software on top of all the problems of making software federate, and it’s unclear why everyone who wants work on one set of problems should also be forced to work on all of the other as well.
Isn’t this why libraries exist? Might part of the problem be the insane ambition of people trying to single-handedly make something the size of a browser or email program, WHILE ALSO making something as complicated as a blockchain consensus mechanism that can negotiate in open-world, living-standard protocol with interlocutors that recognize no authorities in common?!?! When you put it that way, it sounds insane that everyone is expected to bite off ALL THIS.
Splitting “making fediverse software” off from “making smooth interop across moving-target protocols” would be possible if core libraries like fedify handled not just the de/serialization of the JSON-LD wire format but also the S2S mechanics which are… really the only other part where you’d need to know what JSON-LD is.
Forgive me if this sounds insane, but I think that regardless of whether the core spec and/or its “authorities” consider the spec fundamentally JSON-only or JSON-LD, whichever side of the holy war you consider the “extension” and which the core, the functional outcome either way requires the extremely finicky crap to be outsourced to a blackbox library anyways.
I.e. we might be fixated too much on who’s right and what the spec says and not enough on what the spec does, because until the library layer and the software layer are distinct and there are “federation libraries” in a few languages, it’s not going to be easy enough for developers to realistically make good federated software anyways.
i don’t think it quite reaches the level of “holy war”, but the fundamental conflict here is ironically how to resolve conflicts in a decentral manner, given that this is an open world system. that is to say:
the “common ground” is that we at least want to resolve conflicts, right? hopefully? it’s just a question of “how much coordination is really required, anyway?”
i am generally sympathetic to the argument that we can just resolve it all via social means, but my concerns are twofold:
first, if you have to use “social means” to resolve the conflict, maybe you haven’t actually resolved the conflict, you’ve just offloaded it. you’re picking one party that gets the rights to use the shorthand term, and all other parties have to find a new shorthand term.
why?
it feels like there’s an implicit assumption that all of fedi or fedidev is happening “in the same space”, that the end result is a single coherent protocol that everyone shares… which is not going to be the case. to force this to be case is to force a single coherent worldview on everyone. a single data model, set of abstractions, and so on. it doesn’t make sense to have the “common ground” be overly specific as to cover things like “statuses” and “accounts”, because for as many softwares as there are where that abstraction might make sense, there are probably just as many softwares where the abstraction doesn’t make sense. instead, we should realize we already share a common ground, in that we are doing all this on the Web. we’ve got rough consensus up through tcp and http+dns. there have been historical debates about whether to use plaintext or xml or json formats, sure. but even if we say we’re only interested in sending json over our http requests, specifically as2 json, that still leaves the shape of the as2 payload completely unspecified.
this term naming conflict thing is like trying to enforce a global namespace on usernames across all of fedi, except instead of usernames it’s term names. don’t we already have a solution for this problem? qualifying a name by giving it a namespace, like a domain or uri prefix? this is a Solved Problem. we have decades of prior art in web architecture that we can draw upon. i feel like we don’t need to poorly reinvent namespacing when we can just use proper namespacing. yes it might be complex, but it’s a necessary complexity. there are consequences to ignoring it. just like there are consequences to ignoring http caching headers or http status codes in current fedi. if we paid attention to http caching headers, both setting them and respecting them, then maybe we wouldn’t have to carpet-bomb all of fedi with Delete activities on the regular. if we paid attention to http status codes then maybe we could understand that an http 301 means a permanent redirect and we should update the id we have stored for the thing we just tried to fetch, no Move activity needed (which is a misuse of the Move activity imo). etc, etc, etc. – it’s all necessary complexity if you want a good UX and a system that actually works consistently. and it’s pretty much all stuff we can solve by leaning into the web architecture instead of trying to fight it or ignore it.
second: how do you use “social means” once fedidev grows beyond a certain point? i think the idea that you can just reach out to any other implementer and resolve a conflict is something that stops making sense eventually. if there are 10 implementers, every implementer has to talk to 9 other implementers. if there are 100 implementers, every implementer has to talk to 99 other implementers. what then? or: what if out of the 10 implementers, 3 of them won’t talk to you? what if out of 100 implementers, 33 of them won’t talk to you? it doesn’t even have to be hostility. it could be simply ignoring you, or maybe they don’t see you as “worth their time”. isn’t this what people are worried about with EEE concerns? that the big companies get in on this and eventually stop including all the smalltime devs in their protocol talks? or that they start making incompatible changes and everyone else feels compelled to play along?
the problem with “social means”: a concrete example
even in the case where “social means” might ostensibly work, it can lead to undesirable outcomes.
say @silverpill introduces a FEP describing Conversation Containers as implemented by (streams), which i think is something they’re working on. they define a term Context which means “this collection is a Conversation Container”. well, say i also have a FEP that uses the same term Context to mean “this collection Adds objects that share the same context”. is this the same usage? no, it’s not – the latter doesn’t imply support for Conversation Containers.
so now we have a conflict. some other dev wants to implement both our FEPs. how can they do that?well, in a world where we don’t bother with @context, we could just use the full IRI:
but this requires us to agree ahead-of-time that “the network” consists of everyone who agrees to “the protocol” that we simply never define terms for our extensions.
we might not agree. the other dev in this example might decide they want to ship their own @context anyway, tailored to their specific application domain:
so now anyone wanting “full support” for the extension has to, as AS2 says, support curie and expand Foo and Bar to their full IRIs. if they are okay with just “partial support”, then they can choose to only handle cases where the full IRI is present, and drop any type that they don’t understand. or, in the worst case: maybe they hardcode Context in their application to mean one specific FEP over the other, which they might not be aware of. this is the kind of fragility we want to help developers avoid entirely.
say that after some point, @silverpill decides to change their FEP’s term Context to Conversation… but after that, i introduce another FEP that also uses Conversation, and instead of referring to Conversation Containers, it is defined as “a collection that contains only “post” objects, i.e. objects with content”. this definition is incompatible with Conversation Containers, because those can contain activities without content. we end up in the same situation where some other dev might be aware of one or both FEPs, support either or both of them, with or without a term definition. see how it gets messy?
alternatively: say that @silverpill introduces their FEP, some months pass and it gets implemented by one or two softwares. i end up not being aware of this for whatever reason, and i come in months later with my FEP that uses the same term. am i just locked out of using that term? what if i use it anyway? well, now we’ve created incompatibility.
this practice is known as “namesquatting”. it holds that whoever is first to using a term can prevent others from using it without consequences.
synthesis: who’s afraid of the big bad json-ld?
basically, if we want to avoid JSON-LD, we have to pick one of the following outcomes:
enforce across “the network” that “the protocol” bars anyone from using their own @context, and the only allowable @context is The One True Context (AS2). use full IRIs, always. if the extension gets popular, it can be proposed for adoption into The One True Context, at which point you get to namesquat your preferred term definition. (did i mention that you can’t use other specs like ODRL or Web Annotation because that would confuse the plain-JSON parsers?) (did i also mention that updating The One True Context technically makes all your old AS2 documents noncompliant? because those old documents used IRIs, and to be consistent with compaction against the new context document, they need to use whatever the new term is!)
fracture into many different “networks” that partially overlap, where each “protocol” is defined by whatever context you share with any other implementer’s context, explicitly or implicitly. so there is the Mastodon Network where you are assumed to share Mastodon’s context, etc etc etc. (oh hey, this is kind of what we have today for LD-unaware implementers that neglect to or refuse to implement CURIE support!)
note that when i say “shared context”, i don’t necessarily mean an explicit @context declaration. i mean literally, a shared contextual understanding for what terms like discoverable or indexable or featured mean. this can be implicit, and indeed, in “plain JSON” you can be said to have an implicit context at all times, baked into your software by virtue of what you choose to understand and implement ahead-of-time. put another way: not only is JSON-LD an extension of JSON, but at the same time and in equal measure, JSON is just JSON-LD with an implicit context. more specifically: a JSON object is just a JSON-LD document in compacted form where the @context is not declared explicitly.
libraries won’t save us (but c2s will)
libraries only get you so far. the problem is with the monolithic model. if your program implements everything from scratch or if it instead outsources that work to 1000 different libraries, then… isn’t it still doing too much, regardless?
as “simple” as s2s can be, there’s a lot of machinery and plumbing to it that i think is ridiculous to force on every new implementer. using a library can help you do it faster, sure, but you’re still doing it. you’re saving duplicate work by depending on the library to do some work for you, but you’re still duplicating work by not depending on a completely separate server program to do that work instead!
it’s like if Outlook had its own SMTP server embedded in it, and Thunderbird had its own SMTP server embedded. and Geary had to embed its own SMTP server as well, and Claws Mail embedded its own SMTP server, and so on. isn’t that ridiculous? why have every single email client also be an email server? just have one email server and connect to it from different email clients.
what’s really causing developer burnout?
well, this is what “the spec” “does”:
define a Client profile
define a Server profile
define a Federated Server profile
guess what people ended up doing? that’s right, none of them!
what you refer to as “the library layer” and “the software layer” actually maps pretty well onto the Server and the Client. here’s the basic idea:
the Server is for plumbing all the social bits. it’s where storage and delivery happens. if there were a library that let you do all manner of black box things, then the Server would need to pull out the “AS2” box, the “C2S”, and the “S2S” if it happens to also be a Federated Server.
the Client is where your business logic goes. it’s “the software layer”. got an idea for a cool app that uses activitypub? cool, make it a client. outsource your activity delivery. there’s no good reason to duplicate that work. spend less time thinking about the mechanics of how Servers accept and deliver activities, and spend more time thinking about how you can use those social bits to enhance your application.
this is the rub, innit?
it all just boils down to the age-old refrain: “we’ve always done it this way”.
you’re absolutely right that expecting devs to get into the ecosystem is a herculean task, and that many have not succeeded. many have burned out. many have given up.
because the default is to not only build an entire web browser, it’s also to build an entire email server. from scratch. well, maybe there are libraries that can help you do it faster now, but these are still two separate concerns that need to be separated. but they aren’t.
“to bake an apple pie, first you must invent the universe.”
how do we move past this? by standing on the shoulders of giants, instead of standing on each other’s feet. we don’t have to solve every problem from scratch. divide and conquer.
it’s why i believe very strongly that a reference generic activitypub server ought to exist. without that, all we have is a spec.
and if there are any missing bits that prevent a reference generic activitypub server from existing, then these ought to be direct inputs into the vNext effort.
Either this thread has gone massively off-topic from listing desired changes.
Or it listed the single one biggest pain point that exists in AS/AP today.
IMHO it is the latter. The specs say “We are linked data JSON-LD notation, but you may use plain JSON”, and subsequently the developers in the Fediverse ecosystem decided they preferred plain JSON. Linked data solutions are the exception, not the rule.
Linked data as a suite of open standards in general is not a well-trodden path. It has weak adoption and tool support, and specs themselves are still evolving. After many years linked data just isn’t there yet, hasn’t fulfilled its promises. And pushing its complexities through people’s throats while they do not see its merits only helps turn them away.
AS/AP must make a choice. The either/or pick’n choose of JSON-LD vs JSON makes no sense.
If you say something is linked data, then there’s N open standards it should comply to. If people can consider plain JSON they’ll not care for any of those. You get both crappy linked data extensions as well as plain JSON extensions that way.
What Linked Data is best suited for is as a host of standards that allows any data to be marked up as information so that tools in a best-effort manner can reason about and infer its semantics. As @trwnh says this allows vast open world knowledge networks to be created. Undoubtedly where crawlers, reasoners, inference and semantic query engines plus AI make the most of this semantically hyperlinked data.
IMHO this is not what is needed for the AS/AP communication protocol of the Fediverse.
It may be part of what is needed, as message exchange between actors using the email-like addressing mechanism also forms graphs in the content that aggregates at any endpoint via inboxes/outboxes.
But for modeling robust heterogenous interoperable services on the Fediverse the message exchange itself is most important. In other words AS/AP is first and foremost a message-based communication protocol. (And it may even be that it spans up an event-driven architecture, where each activity sent indicates something that already happened).
Interoperability i.e. interfacing to a remote endpoint requires:
Data model and message formats that are supported.
Information about the business logic related to 1).
Information about message exchanges that are supported.
It is not enough to just define extensibility for the msg format, which is the extent to which JSON-LD helps us currently. If we’d had JSON Schema and some namespacing mechanism it’d be just as good to satisfy point 1) or another option may see use of Linked Data Modeling Language as I suggested before.
But point 2) and 3) aren’t covered by any standard. In order to do these well, eventually there’s good quality documentation that should be available and discoverable. And its quality and value depends on its level of interoperability (custom/app-specific vs. standardized), which in turn depends on collaboration and finding agreements within domain-specific ecosystems that develop particular servives.
There’s OpenAPI and AsyncAPI. For the exensibility mechanism of the AS/AP-based Fediverse we might have federated specifications, whereby each endpoint / actor offers the means to discover its interoperable specification.
Throwing it out and replacing it with something that actually supports extensions for both LD and non-LD consumers helps everyone. Using JSON-LD for extensions is not useful for non-LD consumers, which represent a huge majority of AP fediverse software. There are other options that should be considered. If JSON-LD (+ schemas/ontologies) turns out to be the best extension solution, then we should stop trying to sell AP with the “just use plain JSON” narrative that leads to so much developer pain and confusion.
JSON-LD is explicitly an RDF serialization based on the RDF data model (a graph model). Arbitrary JSON is not that and is a tree model. Even worse, the current spec is based on JSON-LD 1.0, which does not allow lists of lists so it’s false that any JSON can be represented in JSON-LD 1.0.
All JSON-LD is JSON, but the inverse is not generally true.
And there’s a difference between an implicit context and no context. The latter applies to most regular JSON data.
If AP content is JSON-LD, then it is RDF. At least some of the spec authors don’t want to accept that, so they say it’s just plain JSON that is compatible with JSON-LD, must conform to JSON-LD compaction algorithms, and uses JSON-LD for extension support. But it’s just plain JSON. Why would that position be confusing to anyone?
extensibility is just part of this ongoing discussion. the other is a lack of C2S.
i’d also throw in “insufficient focus on web resources and access control”, but that’s somewhat adjacent to extensibility, on account of it being possibly handled by extensions like Web Access Control. although there’s a fair amount that is actually in-spec, like “fleshing out groups”. or stuff that is literally in-spec already but being neglected, like the bit about how you SHOULD respect http caching headers. or indeed the bit about how you MUST support CURIE if you want to fully support jsonld extensions, to bring it back full circle.
this is missing the point, which is that in actuality, they decided they preferred compatibility with mastodon over compliance with the spec. in doing so, they decided that they want to be part of “the network” instead of being part of “the web”.
in most cases i would say the json vs jsonld debate is actually a massive red herring. the real issue is the architecture of “the network”. as well as its de facto “protocol”, which is less of a “protocol” and more of a patchwork quilt of partial compatibility.
at what point do we recognize that a lot of fedi’s problems come from the “instance” model? from doing the bare minimum compliance, from implementing the bare minimum of functionality? well, that’s for another thread.
so what is? i mean, you say “AS/AP communication protocol”, but what does that even mean? it seems to me like fedi doesn’t even particularly care to implement AP broadly. AS2 maybe, but AP so very often gets shoved into a Create Note shaped hole that it’s become a meme.
but the message exchange is largely undefined. just about the only restriction is that, if something arrives in your inbox, it is supposed to be an Activity. so you get a Create Note maybe, but nothing actually specifies which properties might be expected or required on the Note.
i’m going to say this again: it would probably make sense to formalize some common profile of how various projects currently federate and interoperate, insofar as one believes this to be a worthwhile endeavour. take The Mastodon Network and turn it into a FEP. but be warned that this will entrench a very specific interpretation of what a federation protocol ought to be. it’s not something i feel like doing at the moment, although i’ve considered it in the past.
The biggest problem I see is that at least one of the AS2 Rec authors is still strongly pushing JSON-LD as an extensibility solution for non-LD consumers in spite of the obvious issues and extensive feedback from AP developers.
. I literally joined this forum as I started an ActivityPub side project and started reading about and learning ActivityPub. It’s obviously very early and I don’t know where it will lead but I am actually trying to implement ActivityPub. The more I implemented and the more I read in the spec the more frustrated I got at how complicated it was. Especially JSON-LD was frustrating as again, I had never heard about it before and it just seems like a no-brainer that a federated, widely used standard such as ActivityPub ought to use the lingua franca that is JSON, and not some weird dialect of it (i.e. JSON-LD). And yet it does use JSON-LD. Sigh.
AS/AP must make a choice. The either/or pick’n choose of JSON-LD vs JSON makes no sense.