(This is a copy of my comment to the Fediverse Developer Network on Matrix)
I quoted this often:
“Any decentralized [ecosystem] requires a centralized substrate, and the more decentralized the approach is the more important it is that you can count on the underlying system.”
fedi is honestly a rube goldberg machine [but] any attempt to do “proper activitypub” will surely create a third network.
One based on real lessons-learned and with the highest potential to be at least somewhat backwards compatible or easily bridged by those still on “mastoverse”.
Yesterday on HN the artice “Why I’m betting on Nostr” was discussed. Now I’m still not that interested in Nostr, esp. seeing some comment that “on Nostr they only talk about Nostr and cryptocurrencies”. Having a greenfield technology playground is a relief, but it can also be a trap when the difficulties come later on. What Nostr seems to have going for it are its simplicity at the core protocol and a FEP-like extension process working from the very start.
We have roughly two approaches:
Wade into humongous tech debt and gradually fix rube goldberg machines in a moving grassroots environment.
Bite the bullet, take lessons-learned and be unafraid to go for a “proper activitypub”.
Both of these approaches can be followed independently, while still benefiting from each other. Both need dedication and committed people to drive them. Though 1) can be ad-hoc and is sort of what’s happening already. Approach 2) I predict is what will happen once big corporate players start to really play our game. We can wait for that, but then it is also these players pulling most of the strings.
There actually is a protocol underlying the Fediverse. I’m unsure how much it has in common with ActivityPub. All one would need to do is document the “Fediverse protocol”. Unfortunately, that’s a hard ask. It requires even more skill than understanding the protocol underlying the Fediverse.
So, all I can say: Neither. Document the current protocol.
Edit: I just saw this issue , I wrote a while ago. I think it describes one of the things, that needs to be documented. I already fail at the first task, I mention: Give a catchy name to the behavior. I think if I name this “the obnoxious actor webfinger link”, most people will know what I’m talking about. Most people will also dismiss my feature, because the name does not fit in what they want from a protocol.
I mostly agree with @helge but I don’t think there is one protocol underlying the Fediverse. There is a closely-related family of protocols that can interop to some extent. However, a large majority of the AP-related Fediverse (by instance and user count) uses the Mastodon Microblog Federation Protocol so that would be a good candidate to document first.
The FEP is a good approach. Best we have currently, although it can be very much further improved.
I personally don’t think we will get rid of tech debt anytime soon with this approach. AP became a W3C Recommendation in January 2018 and we’ve had near 6 years of trying since that time. Tech debt has only grown. Not just that, but post-facto interop (“follow the big projects”) has become much further engrained, steering fedi in a particular (Microblogging) direction.
Wrt a possible effort to define a “proper & minimal AP” … I feel it should be a parallel project, and not hamper any effort done with the FEP and in other various developer hubs that try to bring the fedi to a higher level.
(To any reader… the link to matrix in top post will give you background discussion taking place)
Looking around, I see a fair bit of both (1) and (2) happening side by side.
The best example of (2) in my mind is the emerging test suite.
I’ve seen this tutorial resonate very strongly with anyone who runs into it, and I think it deserves more widespread attention:
A dev partner of mine was having a hard time grokking AP in the midst of developing a Matrix-based app with all the mental mapping that entails. But with Seb’s tutorial it ‘clicked’.
OAuth/OIDC has a similar resource:
For people getting started with ActivityPub interoperability, I see the formal spec as something to be aware of, but not the place to start engaging. You should start where the “action” is at, and right now that’s Mastodon in the domain of AP microblogging and Lemmy in the domain of AP forums.
Isn’t this where the “doing-too-much” combination of AS/AP eternally leads to confusion? Where it is more accurate to say that the current intertwined AS/AP et al constitute more of a framework (as Dennis Schubert of Diaspora* mentioned years ago)?
There’s a transport protocol and minimal API. And then there’s an extension mechanism that allows us to define information exchange, which happens to be based on Linked Data.
From that perspective there’s no Mastodon “Protocol”, but a Mastodon Profile. And there are other Compliance Profiles that specify message exchange patterns and formats for other types of app. I should be able to choose and combine these profiles and implement them from their spec to ensure interoperability. But the profiles are a layer above the transport protocol.
I wish we could sit down and figure out the common denominators, clearly separate what belongs to the AP Spec and what does belong to the AS Spec. That would certainly be helpful to understand implementation profiles and where they err (if they do). Note that we could use something like they have on Meta to simplify documentation.
One disadvantage with the ‘backlog’ of this forum is that its a humongous effort to categorize it all, and then subsequently distill the valuable stuff from often very long discussion threads. Before doing the effort we should know if there’s the will and commitment to start the initiative, or be content with approach we already have: slow-growing grassroots evolution in the direction of most active, best established parties.
Update: @j12ton chat said he will create a list of objectives, from where we can distill a roadmap.
I’m really excited to see whether the RIPE NCC Community Fund 2023 will come to us, because this is the kind of work we could pay for with that money.
Also @mro made some post today in the SWICG mailing list that seems related IMO.
On the contrary, I think that Discourse is very well made to capture bits of discussion into consolidated topics that can then become knowledge. Where mailing lists are linear and most forum software follows that model, which ends up with deeply hidden bits of distilled science, Discourse makes it very easy to select and move posts, quotes relevant parts into growing pages, and so on. But I agree this is extra work to ensure wisdom scattered across replies go back to the first post so it becomes useful to newcomers.
If it’s not AP/AS2, what is the transport protocol and minimal API? Is this the new thing that might be defined at some point in the future (that’s been discussed on Matrix)?
I suppose it’s difficult to know what it is when compared with this hypothetical minimal API, but Mastodon has definitely implemented an application-specific protocol that is only partially (some might say slightly) based on AP/AS2. It can be generalized into a specification profile for other applications to implement if they want interoperability with Mastodon. In that sense, it can be both.
What distinction are you making between protocol and profile? The C2S and S2S profiles in the AP specification are defined as subsets of the overall protocol definition (as another example of being both).
Yes, this is related to the chat of the last few days. It is not really about new things that are different than AP/AS2 rather than providing a clearer separation of the various parts of the spec. Pure transport protocol, API and then the extension mechanism where message formats and their exchange patterns are defined.
Current AS/AP are more a framework than a comprehensive spec where you can say “Just implement AP and be part of the fedi”. With AS/AP you get some information, and then you have to figure it out and any spec-compliant implementation can be wholly incompatible with the next one. Dennis Schubert had similar concerns years ago. I’ve dug up those articles:
(Note: They were badly received by many people passionate for AP, but there’s good points in the critique)
With profile I refer to here is ‘Compliance Profile’. Defined by the extension mechanism and what makes my app e.g. a Microblogging or a Cooking Recipe app, or whatever, if I support that profile. It is like the information layer of the networking stack and the transport protocol (you have an inbox and an outbox, etc.) is below that.
Terminology is a thing and different terms may be better. @helge already mentioned that the term ‘extension’ isn’t appropriate either.
Oh btw… “Replace” in the topic title refers mostly to not being afraid to make backwards-incompatible changes, if that greatly improves the spec and ease of adoption.
I don’t think profile is a bad term in this case, but protocols and profiles are often closely related in networked application specifications.
However, if we decide to document the Mastodon protocol as a microblogging compliance profile (with which Mastodon will be automatically compliant, by definition), I think it would be good to be clear about that.
One thing that Compliance Profiles will enable is that work can be decentralized and different developer communities can make sure that their own compliance profiles are in order.
Mastodon’s own community would be the best candidate to care for a Mastodon Profile. Would they also care for a more standardized Microblogging Profile in a community together with a number of other microblogging apps? Possibly.
The scope of this topic isn’t to write the Compliance Profiles. It is to provide a clear method for anyone who wants to do so. (At least that would be my proposal).
In response to the feedback issue on the strategic plan draft, I wrote the following…
@j12t: I define the Fediverse as all the things that interoperate, so the scope is simply all apps that interoperate, regardless what their feature set is, and whether or not they do anything we would call “social networking”.
That definition is still vague. My feedback wasn’t well enough in context either. If we have in glossary:
Social Networking: Any interaction of people using online applications or services.
Fediverse (technical viewpoint): A decentralized social network based on open standard protocols that allow applications and services to interoperate.
ActivityPub: A pluggable protocol that allows application and service developers to define how their solution interoperates with the Fediverse social network.
Then the scope is something like:
This initiative involves taking the next step in the Fediverse evolution. We focus on ease and comprehension of the ActivityPub specification and provide hands-on guidance for application and service developers to join the Fediverse and build standards-compliant and interoperable integrations with other peers on the rapidly growing decentralized social network.
Regarding “Right level of interoperability” I commented…
I think this boils down to ActivityPub being a layered protocol. Also in my latest comment I defined ActivityPub as a “pluggable protocol”. How we define and name the layers should bring clarity here.
Lower layer: How basic secure communication between peers (Actors) takes place.
Upper layer: Where I define the pluggable integration that allows my solution to be part of the Fediverse social network.
With that you get for the scope:
Lower layer: Precise specification, MUST-implement for compliance.
Upper layer: Prescription for interoperable integration, MUST-follow guidelines for compliance.
Overreach: Describe a particular integration (e.g. Mastodon Compliance Profile)
This exercise is left for decentralized community hubs and individual projects.
The point about overreach is crucial. Many dev-related discussions relate to tech debt in a particular domain.
This isn’t or shouldn’t be a “far-future” undertaking. It is what we have already, but in a mixed up way that serves eternal confusion in the dev community, and combined with a discussion about existing solutions that obfuscates the Vision for the Fediverse and its potential that can be unleashed.
@helge: A source of trouble is ActivityPub not being great at being a specification document, one can base implementations on. This can only be remedied by replacing ActivityPub through one of the official channels, or by abandoning the official channels, and declare oneself on of the official channels.
Either way you end up doing specification work.
@j12t: Agree on specification work, definitely needed. Just not the form of specification that is intended to become an official standard. That’s the domain of the W3C, ietf, iso and what have you.
@aschrijver: It might be part of the FEP, which then serves as sort of a ‘staging ground’ for stuff to become official later on. I am not fully caught up with discussions on FEP process improvements… but we once discussed having a kind of separation between like ‘core’ FEP’s and ‘other’ FEP’s (rn anything can be FEP’ified and enter the same growing list, like MyYummyYummiCatRecipes FEP).
Core FEP’s might be chronologically numbered, and other FEP’s have their generated ID’s.
The core FEP’s should become as thorough as that PubSub XEP. They have higher quality requirements and maybe more formal process to them.
How that might look like in terms of an Integration Guide…
Welcome. This guide will take you on a tour to integrate your project with the Fediverse. In the first part we will guide you through the Fediverse Enhancement Proposals (FEP’s) that alllow you to implement the parts of the ActivityPub protocol that are minimally required for compliance. We will steer you through the W3C open standards. ActivityPub is a pluggable protocol, and the second part of this guide provides a methodical approach and best-practices to develop interoperable extensions in support of your Social Web use cases.
@trwnh: this is a statement that needs more explanation – what happens to the old generated id? what determines if an FEP is “core”? and “core” to what and whom?
And my take on this is:
I don’t know if “core” is the correct name, hence the scare quotes. There are some things in the specs that any AP impl for any type of app must minimally take into account to be compliant. The MUST’s in the spec, if you will, but also some additional stuff. The holes in the spec around security, etc. Other stuff that’s still unspec’ed ‘tech debt’. Must-haves to securely exchange msgs between endpoints (actors) on the Fediverse.
This “core” may also include some new stuff we add in order to provide a stable, comprehensive foundation. More stuff around security most likely, nomadic identity maybe, and possibly also ‘compliance profiles’ that allow devs to build heterogenous integrations on top of the core. Define the extension mechanism of the pluggable AP protocol, if you will. As said, that can be very minimal at the start… I just want to find the specs for your list of supported use cases.
Then there are those things - many of them also ‘tech debt’ - you must do to interop with a subset of app-types / domains. Here’s where many very specific have organically grown over the years, and they involve the Mastoverse and increasingly the Lemmyverse/Threadiverse. This stuff is either Compliance Profile scoped, or they may be optional “core” FEP’s, for instance.
If you look at the XEP’s like XEP-0060: PubSub they don’t just have a status going from ‘Experimental’ to ‘Stable’, but they also have a version field.
If we’d go with a similar scheme for AP, then in the Integration Guide we get something like:
As for the existing FEP’s and their numbering… they are kind of all over the place in subjects they address. We defined a very loose, informal process. At least for “core” FEP’s we should go more formal. Some current FEP’s may be candidates to become core, or have part of the info to be moved to a core FEP. The remaining FEP’s can continue to use their generated ID’s.
So a Compliance Profile might be something like this (just brainstorming):
- name: Microblogging Profile
- id: PROF-ad3cf
- description: This profile encompasses a minimal feature set for Microblogging apps on the Fediverse.
- supports:
- FEP-0001, FEP-0002, FEP-0003
- FEP-0006, FEP-0009
- specification: https://activitypub.rocks/profiles/microblogging/latest/
--------
- name: Mastodon Profile
- id: PROF-13af5
- description: This profile offers the full range of features that Mastodon provides.
- supports:
- PROF-ad3cf
- FEP-874aa, FEP-6bf31
- specification: https://docs.joinmastodon.org/spec/activitypub/
Just to sketch the idea. Idk if profile dependencies are a good idea, and maybe there should be an extends property or whatever. And instead of PROF-xxxxx the ID’s could be just FEP’s too.
For the record, I agree that there should be steps taken to improve FEP. I would like to see the following:
A standard way for implementations to say, which FEP they support.
A standard way to submit FEPs that does not require the use of git.
A slicker way to read published FEPs.
A slicker FEP landing page.
I don’t have the bandwidth to follow up on these things. In particular, as these are not things, one should just build without consulting all the people that have been involved in making FEP a thing.