FEP-2e40: The FEP Vocabulary Extension Process

This is true for the FEP (as your context includes a SHA256 hash only), but I am at this point thinking of a different proposal, which is to use urn:publicid coupled with an fep namespace that has sub-namespaces for each FEP in particular (fep:5624 would contain terms for FEP-5624, such as fep:5624:canReply). This is analogous to XMPP’s XEP namespacing, such as urn:xmpp:omemo for XEP-0384: OMEMO Encryption.

This part is fine. I don’t have any issues with this; in fact, I think it is a good idea for bundling up all FEPs into a singular context for JSON-LD processors, similar to the schema dot org or the activitystreams contexts. I just think that the definitions shouldn’t necessarily be URLs. In other words, I think urn:publicid:fep:5624:canReply expresses more clearly “this is the canReply property as defined by FEP-5624” rather than https://w3id.org/fep#canReply which instead expresses “this is the canReply property as adopted by the FEP process”. The latter implies some sort of unity among the FEP process which isn’t really there. If we want it to be there, we need to change some things about the process, especially regarding how FEPs are superceded and how property names are defined/adopted:

  • Can anyone supercede anyone else’s FEP?
  • What if different projects adopt different FEPs, such as one project adopting the old term definition while another project adopts the replacement term definition? Do we discourage or disallow using certain names just because they have already been used by a past FEP?
    • Following this line of thought: Is it our responsibility to maintain a consistent vocabulary or ontology? It would be kind of like if the IETF decided that all RFCs must be consistent with each other.

Over on the XMPP side, the XEP process has an “experimental” phase in which namespaces (and names) can change. Taking OMEMO as an example, if we want to fetch the device list for any account, this has over time been defined as urn:xmpp:omemo:0:devicelist, urn:xmpp:omemo:1:devices, and currently urn:xmpp:omemo:2:devices – each namespace change represents an incompatible breaking change in the experimental spec. Sometimes the property names change.

Instead, the FEP process seems to have gone for a process that includes supercession by future FEPs rather than allowing “experimental” FEPs. The XEP has an authoritative namespace (urn:xmpp) registered to the XMPP Standards Foundation. The FEP process has no such thing. The proposal to use https://w3id.org/fep in effect establishes such a namespace for the FEP process, but it does not have any consideration for sub-namespaces. Sub-namespaces for XEPs are managed by authority of the XSF – you can register a sub-namespace like urn:xmpp:omemo as part of your XEP. Sub-namespaces for FEPs are not supported under this current proposal. To support sub-namespaces, the proposal would have to be changed such that it clearly establishes who has authority over the https://w3id.org/fep namespace, and it would also have to use / instead of # in its partial term definition – in other words, define fep as https://w3id.org/fep/ instead of https://w3id.org/fep# so that you can resolve sub-resources.

{
  "@context": [
    {
      "fep": "https://w3id.org/fep/",
      "canReply": {
        "@id": "fep:5624/canReply",
        "@type": "@id"
      }
    },
    "https://www.w3.org/ns/activitystreams"
  ],
  "canReply": "Public"
}

Changing the partial term definition is the easy part. Establishing authority over a common namespace is the hard part. And also, once again, using a URL at all means you depend on w3id.org staying up forever (and maintaining control over it). Otherwise it’s no different (actually slightly worse) than using a URN in the first place (because of DNS/hostname authority).

Yes, it’s nice to be able to take property names verbatim and resolve their definition. But this requires domain authority. Whereas, with a urn:publicid, there is no authority. I should note that urn:publicid is inspired by XML public identifiers, and the urn:publicid defined in RFC 3151 basically allows translating arbitrary strings into URNs. For our purposes, the public identifier fep:5624:canReply contains enough information to both specify a name without collision, as well as find the associated specification and definition ourselves (by searching for FEP-5624). Can you do this with a URL? Sure, if you’re willing to claim the necessary authority and responsibility.

So, if we’re willing to claim that necessary authority and responsibility, I think we should do so in a way that doesn’t expect consistency among every single FEP. Specifically, I would be opposed to URLs of the form https://w3id.org/fep#canReply, preferring URLs of the form https://w3id.org/fep/5624/canReply which clearly preserve the lineage of each term definition. Or, alternatively, we have to work out the rules on requesting that certain property names be adopted as term definitions under the common FEP namespace (so that once a term like canReply is “used up”, future FEPs know how to proceed regarding changing this definition or otherwise proposing a new term).

1 Like

IRIs include URNs. IRIs are just the internationalized form of URIs, which includes both URLs and URNs.

The SHOULD only applies when referring to nodes in a graph.

Which is to say: URNs are valid in JSON-LD. If they weren’t, then DIDs would not be allowed either (since DIDs are URNs).

Only insofar as a project/vendor/implementation/etc is willing to hold itself pursuant to the FEP process. I can imagine projects like Mastodon will want to continue using their namespace no matter what happens here. It will take some very good reasons to convince them to do otherwise. And the process of grandfathering in or otherwise reparenting existing extensions will not be something they want to do.

1 Like

Thanks for your clarification, @trwnh. I misread the ABNF thinking the first part in ihier-part was mandatory (the part having the //).

Still in the DID spec there are no examples of urn:foobar, did:foobar in the @context. They are all formatted as URL’s, and most likely all dereferencable to a JSON-LD document (didn’t test that, but for the the DID Specification Registries it is a requirement).

Yes, absolutely. The best-practice is wrt to broadest interoperability, and Mastodon’s toot is to be adopted as post-facto interoperability. Changing it would be very disruptive. Also any project is wholly free to make their own app-specific vocabs, of course.

Making reference to related Fediverse discussion started by @stevebate :

I think there are two main changes, that I should make to FEP-1570:

  • Rename it to “The FEP Vocabulary Extension Process”.
  • Use @trwnh’s plan to first define terms as fep-id:term, and then possibly include them as just terms. A canonical point for the transition fep-id:term → term would be if the FEP goes FINAL (and nobody objects).

A term of the form “fep-id:term” will be called experimental.

Finally @aschrijver has suggested including something as like Best Practices. Does somebody have some proposals for this? For reference:

i think using DIDs to define the context for DIDs would be a circular definition, no? :slight_smile:

this just has to do with json-ld processing. if you provide a string value in the @context then it is coerced into an @id (and then fetched and inlined into the document for processing). meaning you can type https://www.w3.org/ns/activitystreams and the json-ld processor will do an HTTP GET with the ld+json Accept type, and include whatever is served from there. it has nothing to do with an individual term – any term can be defined as any IRI, and IRIs include URNs.

well, you would want to adopt mastodon’s https://joinmastodon.org/ns# (it doesn’t matter what you call it) only if you cared about compatibility with mastodon and/or found its definitions useful. consider the case where the DID Core terms adopted alsoKnownAs from activitystreams – DID Core v1.0 Terms shows that the DID @context includes the security and activitystreams vocabs as well, and the only term adopted from activitystreams is https://www.w3.org/ns/activitystreams#alsoKnownAs. if you really cared to formalize some prior term, you could adopt it in the same way… but do you have the authority to (re)define it?

the problem with relying on mastodon’s namespace and terms is that you are bound to their expectations. for example, if you wanted to use featured for your concept of a “featured” collection, do you include only “posts” or do you also include “hashtags”? mastodon defines a separate featuredTags collection currently, but they might unify the two properties at some point and even integrate the concept of “featured accounts” into the same featured collection. or they might do something completely different. who knows? you’ll have to deal with that uncertainty that mastodon might change the meaning at any time.

luckily, there’s not actually much in there that you “need”. mastodon has been somewhat careful in adopting existing vocabularies where possible. maybe even too careful – the “profile fields” feature depends on an (incorrectly mapped) term from the schema vocab, and it could have been implemented using native activitystreams terms. see FEP-fb2a: Actor metadata for more details.

mostly, you might care about adopting Emoji for custom emoji, or blurhash for more colorful placeholders, or focalPoint for centering part of an image when using cover-fit. maybe you want suspended for marking an account as locked by moderators. everything else could possibly be better defined by some other vocabulary or FEP:

  • discoverable could be replaced by a fuller-defined set of flags indicating consent for various data representations and usages. (as it stands, this property is currently only used for the somewhat-defunct “profile directory” feature and also for the “trending posts” feature.)
  • votersCount is dependent on implementing polls in the same way as mastodon. (i’d honestly recommend or possibly take on myself a more “proper” definition of what polls should look like – the as:Question type as currently (ab)used is not really fit for this. but then again, it was based on a non-normative section…)
  • featured and featuredTags have the problems described above (their contents are unspecified and their definition is quite mastodon-specific)
  • IdentityProof is no longer used but i suspect there’s a better term or really a better way to express this use-case – probably DID verification methods, but i’m not 100% familiar with how those work

i’m not entirely sure of the whole landscape and which implementations have defined which terms and how many of those definitions are actually relevant (given that some implementers are “plain JSON” and not “LD-aware”), but i suspect a non-negligible number of extension properties can be formalized in an FEP or some other recommendation/proposal. it would have to be evaluated on a case-by-case basis.

1 Like

Rechecking the JSON-LD documentation, I realized that Vocabulary has some defined meaning inside it. Thus probably “The FEP Context Extension Process” would be a better name, as what it does is add stuff to the context provided through the fep process…

Any opinions?

You realize that URNs have to be registered, and it’s not a low bar

For content addressable hashes you can use the ni: scheme

The whole point of vocabs is anyone can make one e.g. at w3id.org, doesnt need to be fep/

You also have both context and vocab

I’ll say at this point almost all the vocabs ive seen are fundamentally bugged or wrong, including AS2.0

We may need to start a lot of this from scratch, to do it correctly

Here’s one I was working on lately for user profiles:

http://w3id.org/user

2 Likes

you could just call it “the fep context”? since you’re proposing not only an extension process but also just that we have a context at all / in general

i’m not sure about the other points or how they are relevant

“The FEP Context” also sounds good. I will descope a few things. In particular:

  • Discussing what I meant with objects like “fep-math”, where one defines “group” to be the real thing and values to be things like “monster”.

My reasoning is that it’s not necessary right now and makes things unnecessarily complicated.

So the technical contents will be:

  1. Introduce https://w3id.org/fep and define how it links to jsonld file and markdown description
  2. Allow every FEP to create entries of the form fep:XXXX:name, where XXXX is the FED id.
  3. Define a way to add name = fep:XXXX:name to https://w3id.org/fep

Adding an entry requires to both provide its definition and a description of its usage. Entries of type name should have a broad usage. Otherwise having something like

"@context": ["https://w3id.org/fep", 
{
    "name": "fep:XXXX:name"
}]

is considered preferable. I believe the above still retains many benefits from just name, i.e. linked documentation, clear process, while not causing any “namespace pollution”.

if we’re going with fep as https://w3id.org/fep/ i think this should be fep:XXXX/name so that iri expansion results in a proper url https://w3id.org/fep/XXXX/name. if we’re going with fep as urn:publicid:fep: then it should be fep:XXXX:name so that iri expansion results in a proper urn urn:publicid:fep:XXXX:name. given that you continually express preference for properties’ specifications being accessible via the web immediately, i would recommend the former.

the w3id mapping could look something like this:

  • https://w3id.org/fep accepting ld+json leads to the singular “fep context” that contains all terms
    • otherwise, it leads to some overview page for the fep process or namespace?
  • https://w3id.org/fep/XXXX accepting ld+json leads to an fep-specific context document?
    • otherwise, it leads to the FEP-XXXX document as submitted
  • https://w3id.org/fep/XXXX/name leads to a web page defining the term and linking to the relevant documents

considerations:

  • i float the idea above of having an fep-specific context document. this would be submitted with the fep and then you could just compile a machine-generated unifying context file from all FINAL proposals.
  • i suppose you could use https://w3id.org/fep/XXXX#name if you wanted the definition to be inlined into the fep document itself, but this should be consistent. on codeberg, for example, you would need a header with the exact name of the property which should not overlap with any other headers. this can lead to weird edge cases like, say you want to define a property called requirements but you also have a section #requirements. to avoid all this i recommend/prefer not using url fragments in term definitions.

taken together, i think this would imply something like the following structure in the git repo

context/
- context.jsonld
- 5624/
  - context.jsonld
  - canReply.{md? html? txt? no extension?}

context/5624/context.jsonld:

{
   "@context": {
    "canReply": { // define your property here
      "@id": "https://w3id.org/fep/5624/canReply", // give its full iri here
      "@type": "@id" // if coercion is needed, you include this bit
    }
  }
}

context/5624/canReply.txt (or whatever):

An advisory property signaling that replies from these actors will be accepted. Actors not included in this list may or may not have their replies discarded.

maybe recommend plaintext for these human-readable definitions

I think we are in agreement on the general idea for the FEP. It is also great that you have made a list of some of the properties that are up for FEP consideration.

Also agree with these examples, but other than examplary we need not further focus on the particulars, and just focus on the process itself. There are likely a whole bunch of cases to be considered afterwards once the process is in place.

You mean by means of the @vocab keyword? And the mention of Default Vocabularies?

I think “Vocabulary” is appropriate. For instance the JSON-LD spec also mentions:

For example, the prefix foaf may be used as a shorthand for the Friend-of-a-Friend vocabulary.

We also have vocabularies in the same way. The Security Context vocabulary, the Mastodon Toot vocabulary, the ActivityStreams Vocabulary.

A question remains: When is something a vocabulary?

Above I mentioned in par. Human Readable that "Also note that in terms of how FEP’s relate to [vocabularies] there is no 1-to-1 mapping at the moment. And maybe there shouldn’t be one either, so that FEP’s and [vocabularies] (which define AP vocab extensions) are entirely separate.

A FEP might define one or more properties to be added to an existing vocabulary, or multiple vocabs, or introduce a new vocab, etc. Individual FEP specifications in that regard aren’t vocabularies, where each document gets their own namespace. But there still can be one universal FEP vocabulary where particular properties are added that are generic additions to the AP protocol stack.

It gets a bit confusing, but once again I see the aforemention division…

So 2) are docs, mostly related to core protocol extensions that are universal and relevant to any fedi project. And which also define the processes we use, i.e. the FEP process itself, and the Vocabulary process.

And in 3) we may have a FEP vocabulary collecting core extensions in its own @context and I like the ideas of @trwnh in the comment above, but we also get a growing set of other vocabs that are often used and their use increases general interopability, albeit sometimes in very particular domains. In this list of vocabs Mastodon might also submit their @context doc pointing to their own domain and docs location (example of a post-facto interop adoption).

It is the Process that we are defining now. How many vocabs there are and how they are named and defined is something that is a second step, imho.

But I think there’s more than just a FEP vocab (though this one is our starter for what we have already been specifying in prior FEP docs). Like e.g. I could well foresee a need to have some vocab around video platforms soon as more video-related apps are making their entrance to the fedi. Another example is the Podcasting scene that is working on their own vocabulary. See: activitypub-spec-work.

Though the background and uses are a bit different in our case, I still think for the process we can take inspiration from aforementioned DID Specification Registries document.

a vocabulary is a collection of related terms. i think it makes more sense to say that each FEP proposes its own vocabulary, rather than the FEP process providing a vocabulary. rather, the FEP process might instead provide a bundled context that contains the vocabularies of each finalized proposal. or, with the way i floated above, you could import the context for only one FEP at a time, so you can pick and choose which FEPs you adopt

Yes, but in that case the FEP process needs to be modified to reflect that, as currently it is not about related things, but a FEP might just say “I want to add this one property in this particular location/use case”.

AS Groups is an example. “Here is how groups should work, when we encounter them from the AS vocab”.

This is what I am also trying to say. Maybe should draw a diagram of it all :slight_smile:

An example for FEP-bar… someone submitting a “Video Transciptions” FEP and saying:

  • I have this new vocab proposal for video transcription.
  • But I want propose this type or property to fep-vocab as it is more generally applicable.
  • And to those folks implementing bar-vocab I suggest they standardize this property.

(Note: The diagram can be opened in diagrams.net and directly edited there)

This discussion is currently moving faster than I can absorb information. I’ll take it as a positive sign that FEP-1570 will fill a need. Anyway, I’ve rewritten the first two parts of the FEP and I think they now describe the goals better:

Summary

Current usage of ActivityPub relies on the ActivityStreams namespace [AS-NS]
combined with custom extensions [Mastodon NS]. As far as I can tell, no
best practices exist or a formal process to add new namespaces.

This FEP will remedy this by

  • Create a FEP Vocabulary based on identified best practices
  • Define a process to add new entries to this FEP Vocabulary without a risk of Term collision
  • Define a process to elevate Terms to be common
  • Define a process to create specialized Vocabularies
  • Using [FEP-61CE] as an example how this process can be used.

Background and Terminology

The JSON-LD context is introduced in 3.1 The Context of [JSON-LD]. The context of an object is specified by its @context property.

One can think of the context as defining certain strings to be equivalent. For example Note, as:Note, and https://www.w3.org/ns/activitystreams#Note all represent the same thing. More details can be found in 3.2. IRIs. Following [JSON-LD], we will refer to all three strings mentioned above as a Term.

The second useful aspect of this is that one can define the used terms through the provided URL: https://www.w3.org/ns/activitystreams#Note. Clicking on it will let you easily find the definiton of the Note Type.

We will refer to the combination of Context and easily accessible documentations for the terms a Vocabulary.

The [DID Spec] states Best Practices on how to name Terms in 3. The Registration Process. We will reuse OR adapt them for our purposes.

1 Like

Great, thx! Yes, I’m also spending time I don’t have… I’ll let it rest for a bit and revisit later to review docs.

Merged the latest changes.

2 posts were merged into an existing topic: FEP-a4ed: The Fediverse Enhancement Proposal Process