Improvement to FEDERATION.md convention: Murmurations

Fediversity Fediverse Futures
1

Background

Proposed last year by @darius in Documenting federation behavior in a semi-standard way? proposed the FEDERATION.md convention, and discussed in SocialCG 2020-02-08 was the convention to have a FEDERATION.md in the root of code repositories of a federated application project, to describe in what ways the app implements its federated messages.

FEDERATION.md was a concession. With it the interoperability capabilities of a federated apps can be described by:

  • NodeInfo: A per-server machine-readable definition, with basic metadata about the server
  • FEDERATION.md: Hand-written doc to describe interoperability in detail, side effects, business logic, etc.

Standardizing full capability negotiation in a machine-readable format was deemed too complex.

Since then a handful of projects have committed to the convention:

The Problem

  • As fedi developers we need to know how Fediverse evolves, which AS/AP extensions are in use, and preferably adopt them ourselves or even standardize them (using the Fediverse Enhancement Proposals process.

  • Ideally we want to have a clear overview of all interoperability designs, both standards-based and custom extensions, in use today without having to hop around and search for info across the web.

  • Even if there is a FEDERATION.md document, they are all formatted differently, and it is unclear whether they are kept up-to-date at all. This makes maintaining of a directory hard and time-consuming.

  • The FEDERATION.md convention is not known by most AP devs, can be streamlined, and provide more incentives to adhere to it.

Update: ActivityPubSchema by @Sebastian may be usable for the validation of federation profiles.

The Improvement

A possible solution that goes a step further than informal FEDERATION.md documents, but still stays well away from tackling the complexity of a machine-readable capability negotiation standard, is to use the Murmurations Protocol to describe how federation is implemented in each app.

  • Primary goal is to deliver consistent documentation on how interop can be achieved, in the form of a document template.

  • Secondary goal is to make it easy to discover this documentation, regardless of where an implementation keeps track of it.

The document template could be set up in such way that it can be read as a checklist by someone implementing a compliant app, describing side-effects, validation and other rules.

Solution

  1. Define a JSON-LD Murmurations Profile that is document template to describe AP interoperability support.

  2. Fediverse projects maintain their own profile, and register it with The Murmurations Index.

    • They can use it for their own website to generate reference documentation from it.

  3. A SocialHub-affiliated website functions as the Murmurations Aggregator and generates a Directory of docs.

By creating a profile there is assurance that all projects deliver consistent content. There can be a bunch of additional metadata in it, such as ‘Last updated’ or ‘Release Version’ to gauge relevance. And ‘Issue Tracker’, ‘Repositories’, ‘Project lead’ and other project metadata to ease inquiries and contributions. Etcetera.

Side note: Federated Murmurations

This topic has been split, see also: Federating the Murmurations Protocol

1 Like
2

Hi @aschrijver et al

Thanks for kicking off this thread - it sounds like a great idea :slight_smile:

We would be happy to jump on a call if you like, to explain more about Murmurations, but essentially what you propose sounds like a solid plan to track how federation is implemented in each app.
All you would need to do is to specify which fields you would like to include, as required and optional, in a custom ‘Fediverse schema’ by following the instructions here and submitting it to the Library. We will check it and merge so then Fediverse projects can make Profiles using the Profile Generator and publish / self host these as they like. Then you could set up a SocialHub-affiliated website (as per your 3 above) to query the index, collect the info from the profile files, and aggregate this to generate a Directory of docs.

Regarding your question about federating Murmurations servers, it’s an interesting idea which we have further down our roadmap (we don’t want the index to be a single point of failure), so we would be glad to explore it with you when we get there.

3

2 posts were split to a new topic: Federating the Murmurations Protocol

4

Yesterday on fedi we discussed @grishka’s idea to have ‘compliance profiles’. The idea is appealing and goes beyond what is described in this topic.

Note: Elaboration should be addressed in a separate topic, but I want to make mention of it here.

@grishka wants ‘Per Actor’ compliance profiles:

“[FEDERATION.md / NodeInfo] It’s per-instance. I’m looking for something per-actor, and preferably something that could be combined with privacy settings. Something that, given an actor and activity, could answer the question “can I send this actor this activity”. The answer could be a simple yes/no but also something like “yes but only if they follow you”.”

In addition on FEDERATION.md @grishka finds that “As with many SocialCG proposals, it’s too theoretical. It’s ideas built upon other ideas with what feels like no code written”.

Here I have to refer to SocialCG 2020-02-08 meeting notes, where the rationale was discussed and the decision was made. Some excerpts:

@melody: […] in a lot of ways just knowing the types isn’t enough, there’s a certain point you need the human intervention to understand the semantics of what the types are and how they’re used… […] I think it makes sense to have something a human can read and interpret.

[…]

@nightpool: I think it would be very sad if FEDERATION.md was the only way of approaching this, and I think nodeinfo is valuable as a very broad “survey” of what is currently implemented on the fediverse. But notably, and this is why we originally suggested a simple JSON list of types, is that getting this kind of rich granularity is very, very hard. It’s hard to build this kind of contract language that can describe these features well […]

@melody: […] I love that FEDERATION.md does not try to be that thing, because there’s so much unspecified side effect behavior and etc, and I’m not sure how to represent in a machine-parasable way […]

@nightpool: […] One issue I have with nodeinfo is you can’t do two on the same server… one issue with mastodon is our dependency on fediverse has made it hard to deploy on the same domain (eg deploying alongside nextcloud). I think going forward, especially with the way AP was designed, would love to see things happen on the actor level.

So maybe time has come to elaborate on all this. I think it prudent to keep on 2 tracks: the machine-readable path, and the human-readable one.

@grishka you are most welcome to create a new topic on Capability Negotiation / Compliance Profiles, and cross-ref with this topic.

6

I’d also previously made this package that scraped the HTML specifications to generate an rdfs schema. I find this to be a useful form to consider a schema, since it’s just more linked data.

e.g. if you have npm, run npx actvitystreams2-spec-scraped. Or here is sample output.

It would be harder to scrape the ActivityPub spec html for schemas, but I do think it would be nice to have an rdfs ontology for ActivityPub things, then use that as input to generate json-schema.

Just sharing in case someone else finds it a useful building block. I made it at the time because I saw so many people doing codegen to make as2 libraries in programming languages that benefited from it (e.g. because go’s type system is limited compared to others).

I tried to make activitypub-spec-html to use that rdfs Ontology as an input to render it into some human-readable HTML. I wish that more as2, ap, and all kinds of extensions used owl/rdfs as their ‘canonical form’, then we could have reusable sets of tools to generate documentation, test suites, etc.

1 Like