3-Stage Standards Process: Guaranteeing an open and decentralized ecosystem

Standards Fediverse Enhancement Proposals
1

Open ecosystem, open ecosystem, open ecosystem

In various channels where the Standards Process is discussed there’s mention of a 2-stage structure that goes like: FEP → W3C.

As I proposed many times before (like here, and here, and here, and implied here) I do NOT think that that is sufficient. Simply because it doesn’t match current reality, and will also not correspond to the future that we are able to shape.

On a technical level the Fediverse is decentralized (federated, and in future possibly hybrid p2p + federated), and we all agree that this is major important. Equally important is the realization that the grassroots evolution of the Fediverse by people expanding and improving it, is equally decentralized: The Fediverse consists of countless independent initiatives. This is the social perspective and a reality we SHOULD take into account.

I am for a living standard as mentioned by @melvincarvalho and in line with many people’s thoughts on the openness that is the big quality of the Fediverse to retain.

In any case what we want to establish is:

:point_right:  An open ecosystem that fosters decentralized evolution of the Fediverse.

The current FEP Process is on a good path to gain more traction, but it DOES NOT yet offer the guarantee above. Currently it is positioned to do everything but the kitchen sink: you must write your FEP with us, does not take decentralized communities and devhubs into account.

In order to do so I feel the Standards Process MUST be 3-stage. Reversing the steps as I listed them before, that then boils down to:

  1. AP extensions (msg formats + behaviors for particular use cases) emerge/evolve in-the-wild.
  2. FEP / SocialHub provide a pathway to further standardization and broader adoption.
    • FEP makes AP extensions discoverable, may offer an NPM-like registry for that.
    • FEP Process offers a more formal standards track, similar to what exists now.
  3. W3C standards track goes a step further, up to any formal W3C deliverable.
    • SocialCG and/or WG are in charge. Outcome can be a Wiki page, Note, Recommendation, etc.

Step 1) is the first guarantee that the Fediverse remains a decentralized and open ecosystem not only in a technical sense, but people evolving it for their own areas of interest. Step 2) is open to anyone’s participation, in line with this guarantee.

Transitioning from step 1) to step 2) hinges on a good extensibility mechanism and guidance to adhere to it.

Copied from FediDevs chat. Elaborating a bit on the 3 stage process above…

  1. Someone defines namespace Foo for use in their project.
    • It may be just code in their codebase initially, like in current fedi
    • Spec of formats, behavior is created according to best practice guidelines
    • Foo may be published to one or more namespace/spec registries (compare: apt)
    • Foo matures, evolves, gains popular uptake
    • Foo may become managed by some community/devhub who fosters it for all its use cases
  2. Foo graduates to become part of the FEP process
    • There’s a formal process now, and a range of quality criteria
    • Initiator (project/community/devhub) has authorship and commitment to bring to FINAL
    • FEP teams only facilitate, any fedizen can see their voice be heard by participating
  3. Foo has gained such adoption, it is candidate to for the W3C standards track
    • Foo is taken into SocialCG and/or WG processes, outcome may be any W3C deliverable.
9 Likes
2

Looks good! Potentially gives everyone what they want.

3

I wrote here

All I want from possible specifications is that they come with

  • an automated test suite, that verifies at least every statement of the specification, i.e. MUST, SHOULD, …, once
  • a reference implementation that passes said test suite

I think taking these criteria into account will lead to a very different process than one without.

3 Likes
4

I don’t understand the difference between 3-stage process described here and 2-stage process discussed in another topic. Yes, many protocol extensions are not formally defined and only appear in some implementations, often not documented. Yes, developers don’t need a permission from anyone to extend the protocol. But that seems undisputable to me.

I assume this is the most important part.

I’m not against clarifying acceptance criteria. But changing them to something completely different is going to undermine trust in the FEP process.

2 Likes
5

I think this is the key point . In my opinion, we neither have a good extensibility mechanism, nor do I feel that there will be one in the near feature. A good extension mechanism would have to have properties such as:

  • Is truly decentralized
  • Fails gracefully for applications that do not understand the extension
  • Provides objects that can be safely stored
  • Can communicate that it is being used

The above list is in no way complete, it’s just what came to my mind. I’m pessimistic when it comes to lofty goals, so I just don’t see a mechanism like this existing.

4 Likes
6

Finally, some discussions, like me saying tests and reference implementations, are not really about extensions, but fixing the core protocol. It is hard to judge if something is an ActivityPub extension when half of the ActivityPub specification (= client to server) is unused.

Second problem is the ActivityPub specification does a lot of stuff. Something like #27 - Conversations an ActivityPub Extension - fediverse-ideas - Codeberg.org is needed, but would strike at the heart of a lot of ActivityPub / ActivityStreams specified things. It takes away the CRUD pattern of Create, Update, Delete.

2 Likes
7

First it is about truly recognizing and clarifying this position, so that no one gets the impression that the Standards Process puts itself in a central authoritative position.

Secondly we then actually want to encourage evolution of the Fediverse to be decentralized as well. Suppose there’s a large Origami movement emerging, by all means let them define their Paper Folds extension as a standard in their own developer hub. That needs not be FEP’s that a FEP Editor team is involved with and discussed on SocialHub.

Origami is a weird example, but just to indicate there can be 1,000’s of specialized domains, where domain experts standardize on particular extension specs for their use cases. Here ForgeFed Project and Podcasting-Index are more concrete examples.

As @helge indicates here the task of the “Standards Movement” should be encouraging any practice that increases the ease with which interoperability with these specialized domains can be established. Get above the level where you reverse-engineer someone else’s codebase.

There’s all kinds of help that can be offered here. Best practice guidance is mentioned. A registry of extensions “in-the-wild” that allows you to find stuff in the first place. And possibly “Compliance Profiles” on an (actor) endpoint to enumerate places where specs exist and learn about what the endpoint supports.

4 Likes
8

I get that developers should have the freedom to do whatever they want to extend AS/AP and get their application talking to the Fediverse, and that all of the efforts out there do not need to become FEPs, but then maybe it would be nice to have more widespread support of FEDERATION.md or other structured spec-like information for all of the items in delightful fediverse apps (thank you @circlebuilder and @lostinlight).

And implementers could help the community by posting technical articles about the challenges met and solutions found in the development of their applications, so the community could quickly get ideas without having to reverse-engineer all the source code into spec-ish concepts. Notice of such articles could be gathered here on the socialhub forum and in publications like Last Week in the Fediverse

I know that’s more work that just shipping product, but it would create a more robust development platform with shared concepts that possibly could make their way into FEPs / living standards.

5 Likes
9

ActivityPub and/or Activity Streams 2.0 is a stable core that we can build backwards-compatible extensions on.

Namespaces provide a handy extension mechanism. The security namespace, for example, is broadly used for HTTP Signatures.

I’ve started work on a simple mechanism for incorporating those extensions into the core context. So widely-used extensions come free when you include the AS2 context.

https://w3c.github.io/activitystreams/draft-extensions-policy.html

More AP servers need to support extension activities. We have all you need in the base Object and Activity types to display practically any Activity subtype in a Web or mobile interface: summary, icon, actor. And you can use multiple types to let processors fall back on behaviour they recognise.

Backwards compatibility is key. It requires more care, but the payoff is that you don’t split the network. Most interesting applications can be built on top of AP with backwards compatibility in mind.

1 Like
10

About the ActivityPub API*, I’ve been working on a server here:

And a command-line client here:

I expect to use client.pub to have an Elk-like Web client, too.

The ActivityPub API is really key to making cool apps in the fediverse.

Question for you: how much of the bovine stack runs on the ActivityPub API?

  • I don’t like “c2s” as a name. This is what we call it in the AP spec.
2 Likes
11

From that document

The SocialCG will vote on whether to include the extension in the Activity Streams 2.0 context document.

Not everyone is in the CG and voting is far too easy to rig. A better way might be a living standard, with a diverse and inclusive audience of developers. Then the CG decides through consensus what bits to publish, perhaps snapshots at certain version numbers.

4 Likes
12

Does “The SocialCG will vote on decide whether…” work for you?

I think we typically make this kind of decision with plus-voting; (+1/0/-1). Usually if there are objections, we try to figure out a way to negotiate around it, or usually the proposal just doesn’t pass.

And, yes, there’s a versioning system; if you are really rigourous and only want to use a fixed version of AS2, you can choose that. I think it’s preferred for folks who use Linked Data Signatures.

2 Likes
13

“Decide” is a better term than “Vote”, IMHO. Voting should normally be a last resort, rather than a first resort. But depending on the chairs a vote can be called at any time. And objections can be overuled by vote.

Let’s see if a living standard, with diverse participation and inclusion, is feasible. Then versioning would be like a snapshot of what is being used in practice. With a stable version number to describe the snapshot.

If we can get consensus around the 3 step process. Then the W3C would mainly have things to say about step (3). If done right, it could be the start of a fruitful collaboration.

14

I absolutely support community development of standards around extensions and applications of ActivityPub and Activity Streams 2.0.

I deeply oppose forking ActivityPub and Activity Streams 2.0 core.

  • Formal standards are why ActivityPub is winning. There are a dozen informal social networking standards strewn about the landscape. ActivityPub is popular and widely implemented because we are the formalised standard.
  • Forks are very confusing for developers. You’d be saying to implement an incompatible standard from the one that everyone else uses.
  • Forks cut developers out of the mainstream developer community. Most devs look to the W3C standard for guidance; you’d be misguiding a small number of people. Implementers of your forked standard would have fewer people to work with, less documentation, fewer people to work with.
  • Forks cut users out of the network. If anyone implemented your forked standard, they’d be cutting themselves off from the main body of the fediverse.
  • You don’t need it! If you’re committed to backwards compatibility, which on an active network you just have to be, building extensions and applications covers most of what you want to do.

I’d strongly discourage any fork. Let’s let AP and AS2 form a strong, stable base to the fediverse, and let the innovation happen in extensions and applications.

4 Likes
15

To get back to the original subject: Do we have a 2-stage vs. 3-stage rocket to standardization?

Standards Process

I’m absolutely with @eprodrom that forks should be avoided. Instead there must be more clarity on correct use of both ActivityPub and ActivityStreams, to ensure that implementations converge towards interoperability, not tech debt.

Our current situation in practice is worse than forking, and entails individuals hard-coding flavors of the protocol and not even realizing they introduce Protocol Decay (see RFC-9413). They say “we are part of the Fediverse now”, but only a tailor-made integration allows others to interop.

So AS/AP must provide a solid foundation to build upon. Call it the “core protocol stack” or something like that, which gives federated solutions the means to communicate securely and comes with just enough functionality that any social networking project must take into account.

:point_right:  The AS/AP et al standards MUST provide the minimum requirements for network connectivity.

Then there’s there’s the notion of ActivityPub (by means of Linked Data and ActivityStreams) as a pluggable protocol, where the extension mechanism allows anyone to model extensions for their domain and use cases. This provides federated solutions the means to understand each other, i.e. achieve various levels of interoperability. For this part, imho:

:point_right:  The Standards Process MUST provide clear pathways to ever increasing compliance levels.

We have some mechanisms in place, but they are insufficient. Here we can offer improvements.

Extension mechanism

AP Extensions are message formats (vocabulary) plus message exchange pattern (behavior).

For instance the FEDERATION.md, first proposed by @darius and brought to a FEP by @silverpill is great, an improvement, but does not bring more than an appeal to the developer to “please provide this”.

A stronger version of it would allow making a claim of compliance on your API endpoints (e.g. via “Compliance Profiles” or similar mechanism). Just enough functionality to allow 3rd-parties to discover the specs you claim to adhere to and also say “Hey, I found a compliance problem”. Which either leads to a bugfix or an improvemet to the extension specs.

Linked Data extensibility

Instead of “Compliance Profiles” as described above, linked data’s own extensibility mechanism might be sufficient. Imho there needs to be:

  • An extension specification indicated by the namespace URL. Following the URL gives you the specs.
  • Best-practices to move away from app-specific namespaces.

On the second point one best-practice is indicated in @eprodrom’s draft extension policy: move common formats + behavior to AS2 Extension Registry.

Imho this should only be done if the vocab addition truly belongs in the “core protocol stack” i.e. part of minimum requirements for connectivity. Not doing so would turn the vocab into something like schema.org, a huge ontology with abstract names where 99% isn’t relevant to your use case and still people use all the terms with different semantics and behavior.

Converging towards interoperability

Imho the Standards Process should do everything to encourage convergence into domain-specific profiles. It should provide easy pathways for decentralized developer hubs to emerge and collaborate on extension specs that define the common denominator of functionality in their particular domain / app-type.

Take for instance Microblogging:

  • No convergence, and my app’s @context will list N different microblogging apps custom namespaces, OR (and more likely) it will list the namespaces of the giants in my domain (e.g. Mastodon). And then I add my own custom namespace that the other implementers may take into account, or not.

  • Microblogging apps create a Microblogging community to work on specs. Or they be part of a Moderation working group (e.g. IFTAS). An app developer’s @context now points to the common extension spec maintained by these hubs. Plus their own custom namespace for app-specific stuff.

16

It never really used ActivityPub Client To Server. See FEP-proposal C2S: websocket endpoint for the first necessary modification. Nowadays, no idea. I stopped caring about trying to follow the ActivityPub specification at the end of March, see bovine/specification_template.md at main - bovine - Codeberg.org.

Furthermore, I’m pretty confident that if I further evolve the software to do what I want. The remnants of ActiivtyPub Client To Server support will disappear. Probably also of ActivityPub Server To Server in some sense.

There are reasons, I like talking about the Fediverse rather than ActivityPub. Mostly, ActivityPub in its current form, is not a specification future generations of servers should follow. ActivityPub is too broad in scope. ActivityPub doesn’t have clear boundaries of what is what making modular implementations unnecessarily hard.

Anyway, I consider pixels a perfectly fine Fediverse application. Creating it using Client to Server would be a nightmare.

17

I agree on this sentiment, with emphasis on “in its current form”. Also making this case before were Dennis Schubert of Diaspora* and @cjs (Cory Slep) in respectively these articles:

2 Likes
18

2 posts were split to a new topic: On characterization of the ActivityPub protocol

19

@aschrijver Thanks for writing this, that’s very comprehensive and thoughtful.

On initial reading, I also was like “wait, the 2-step FEP process and the proposed 3-step process are basically the same. the fact that devs can experiment before FEP is a given”.

But on further reflection, I really like that your 3-step proposal emphasizes the pre-FEP developer experimentation, and places attention on the Fediverse putting together really clear and actionable docs on how to create extensions, try them out, and then present them to the larger community.

So, big +1, I’m definitely on board with this whole proposal.

3 Likes
20

What’s the namespace URL, and what are the specs?

Do you have an example (or theoretical example)?