FEP-888d: Using w3id.org/fep as a namespace for extension terms and for FEP documents

Hello!

This is a discussion thread for the proposed FEP-888d: Using w3id.org/fep as a namespace for extension terms and for FEP documents. Please use this thread to discuss the proposed FEP and any potential problems or improvements that can be addressed.

Summary

In JSON-LD, @id is used to identify nodes. Each node obect should contain an @id, which MUST be an IRI or a compact IRI (including blank node identifiers). It is considered best practice in the linked-data ecosystem to have such IRIs be HTTPS URIs that resolve to a definition of the term being used, and it is desirable to define such terms in a JSON-LD context file that is referenced by its IRI rather than having the full @context object embedded in every single document. ActivityStreams 2.0 and ActivityPub do this with the normative context and namespace provided at https://www.w3.org/ns/activitystreams, but this namespace is not generally open to extensions or to experimental terms.

This FEP therefore proposes using https://w3id.org/fep as a base IRI for the FEP process, allowing sub-namespaces for each FEP, and allowing certain terms to be promoted to a default context once proven broadly useful.

2 Likes

I’m reading through it and collecting my comments.

For example, we can consider the definition of “Public” addressing within ActivityPub, represented by the “Public” magic collection. When the normative ActivityStreams 2.0 context is applied, the IRI for this collection may be equivalently expressed as Public, as:Public, or https://www.w3.org/ns/activitystreams#Public.

This statement is untrue, but often made. Due to Public missing from https://www.w3.org/ns/activitystream, only as:Public expands to https://www.w3.org/ns/activitystream#Public. See Public is missing from https://www.w3.org/ns/activitystreams HTML page · Issue #531 · w3c/activitystreams · GitHub.

So please use another example.

		"exampleA": {
			"@id": "fep:9606/exampleA",
			"rdf:comment": "A term with some literal value (string, boolean, number)"
		},

The rdf:comment is invalid. See

An expanded term definition MUST be a map composed of zero or more keys from @id, @reverse, @type, @language, @container, @context, @prefix, @propagate, or @protected. An expanded term definition SHOULD NOT contain any other keys.

in 9.15.1 Expanded term definition.

I would also like to see some rules on naming here. For example, nobody should introduce a term “Twitter” as we do not hold the trademark. I think I adapted from did-core.

Some more thoughts

I currently feel that it would be wise to require examples for each term in the new context and then run:

expanded = json_ld.expand(example)
recompacted = json_ld.compact(expanded, example["@context"]) 

assert recompacted == example

This would guarantee that the new context is valid. Furthermore, these examples will serve to explain the authors thinking. If one uses a build process for FEP documents, one can use this to include the examples in the FEP document.

I actually don’t have a strong preference of this FEP vs FEP-2e40: The FEP Vocabulary Extension Process. Now that I understand what is necessary. Of course, this means that all FEPs introducing Vocabulary terms need to be in the new folder format.

1 Like

that doesn’t sound right – the JSON-LD stuff works on the context document at https://www.w3.org/ns/activitystreams.jsonld which does contain Public being defined as as:Public which expands to the full https://www.w3.org/ns/activitystreams#Public

using the playground at JSON-LD Playground with the following document:

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "to": "https://www.w3.org/ns/activitystreams#Public",
  "https://www.w3.org/ns/activitystreams#cc": "Public",
  "bcc": "Public",
  "https://www.w3.org/ns/activitystreams#bto": "https://www.w3.org/ns/activitystreams#Public"
}

if you look at the expanded form, you see that the IRI forms do not use the definition from context. they get parsed as a @value and not an @id in the expanded form. when they get compacted down against the same context, the as: is preserved but the actual definition is not. so for some reason, the reference JSON-LD parser being used on the playground does not equate a term with its IRI, neither the full IRI nor the compact IRI:

[
  {
    "https://www.w3.org/ns/activitystreams#bto": [
      {
        "@value": "https://www.w3.org/ns/activitystreams#Public"
      }
    ],
    "https://www.w3.org/ns/activitystreams#bcc": [
      {
        "@id": "Public"
      }
    ],
    "https://www.w3.org/ns/activitystreams#cc": [
      {
        "@value": "Public"
      }
    ],
    "https://www.w3.org/ns/activitystreams#to": [
      {
        "@id": "https://www.w3.org/ns/activitystreams#Public"
      }
    ]
  }
]
{
  "@context": "https://www.w3.org/ns/activitystreams",
  "bcc": "Public", // linked node
  "as:bto": "https://www.w3.org/ns/activitystreams#Public", // string literal
  "as:cc": "Public", // string literal
  "to": "as:Public" // linked node
}

the activitypub spec gives this hint:

5.6 Note:
Compacting an ActivityStreams object using the ActivityStreams JSON-LD context might result in https://www.w3.org/ns/activitystreams#Public being represented as simply Public or as:Public which are valid representations of the Public collection. Implementations which treat ActivityStreams objects as simply JSON rather than converting an incoming activity over to a local context using JSON-LD tooling should be aware of this and should be prepared to accept all three representations.


it’s not invalid, it’s just not recommended by the JSON-LD spec. i’m not super invested in having this included though, so i can take it out. i can replace this bit with a link to 9.15.1 Expanded Term Definitions in JSON-LD 1.1 in the absence of any other perspectives.


i’m unsure this is necessary. i understand the reasoning behind it, but it can be useful to use such terms. for example, in the discussion around payment links, it came up that you may want to differentiate between Paypal, Cashapp, etc. without parsing every single href for a prefix. i am not sure what legal requirements may surround something like this


not necessarily. we could set up the redirect rules for the old format as present in the commented out lines. having the new folder format just makes it easier.

https://www.w3.org/ns/activitystreams#Public is completely broken

Click on the link and you get an HTML documentation. It’s pretending to be a linked data vocabulary, but actually it’s not. It’s just a documentation page.

Public is in the context but not in the human readable docs.

AFAIK was all done without change control, so the spec, the “vocab”, and context have diverged, due to human error, there’s no new version number of the spec, and not at all clear how to fix it.

The elephant in the room is that Activity Streams 2.0 is not a linked data vocab, and never has been.

@melvincarvalho What would be necessary to make it a “linked data vocab”? Is the context document at https://www.w3.org/ns/activitystreams(.jsonld) not enough?

Creating a Linked Data vocabulary involves defining a set of terms and their relationships in a machine-readable format, typically using RDF. This allows you to create a structured and interoperable way to represent information on the web.

The important thing is that the vocab (aka ontology) is both human and machine readable. Right now the AS Vocab is only human readable.

You can test this on unix systems with the command:

rapper -g https://www.w3.org/ns/activitystreams

A note on:

https://www.w3.org/ns/activitystreams.jsonld

This is a context, not a vocabulary. I think that is the thing that has confused everybody. They are different things. A context is syntactic sugar that goes in a JSON-LD file to help you save typing out of full urls and datatypes. Generally it can be inline, or linked.

Similarly

rapper -g https://www.w3.org/ns/activitystreams.jsonld

Thows an error

Compare with:

rapper -g http://www.w3.org/2000/01/rdf-schema

This gives back a vocabulary

You are right, I was misremembering details. I’ll reply to the other stuff later, once I have the energy to properly review it.

after playing around a bit more, i managed to get this:

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "https://www.w3.org/ns/activitystreams#to": {"id": "https://www.w3.org/ns/activitystreams#Public"},
  "cc": "https://www.w3.org/ns/activitystreams#Public"
}

to compact to this:

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "cc": "as:Public",
  "to": "as:Public"
}

further expansion or compaction doesn’t result in Public no matter what i do, only as:Public.

This FEP is incorrectly named. It should be 361d

See: Woodpecker

oh, huh, i must’ve renamed it at some point and forgotten to recalculate the slug. i don’t remember exactly what i did three months ago, so i’ll fix it when i next update the proposal.

speaking of updates: i am thinking of removing the bit about “promoting terms to a common FEP context”, as there was a good point brought up about the danger of such an “FEP context” becoming a second normative thing that people don’t fully understand but cargo-cult anyway. this can still be done as a separate dedicated FEP, but it is also a4ed-adjacent, so i’d rather leave it out of this one and move forward with the immediately-useful parts. i’d also rather have such “common contexts” be handled by dedicated FEPs that define conformance profiles, not as something pushed for by the FEP process itself, as i do not believe there is one contiguous “fediverse” that would benefit from having more authority or normativity.

also, the following paragraph:

To facilitate ease of mapping, any files related to FEPs SHOULD be placed within an fep directory within the repository, and FEP-specific files SHOULD be placed within an FEP-specific subdirectory created according to the FEP identifier. [[TODO: is this language necessary to explicitly include?]]

is no longer necessary given that the described procedure was already adopted outside of this FEP.

similarly:

# uncaught:
# - legacy route for assets, e.g. fep/xxxx/assets/file.ext => feps/assets/fep-xxxx/file.ext

is no longer a concern, as the asset colocation adopted should allow resolving such assets naturally. likewise, the alternative routings are no longer necessary or valid, so the whole .htaccess example code block can be simplified significantly.

finally, the bits about extra metadata like rdf:comment in context documents can probably be removed.

EDIT: #116 - update 9606 -> 888d - fep - Codeberg.org contains these changes.

2 Likes

What can be done to move this FEP forward? What to do with alternative proposals (FEP-cb76, FEP-2e40)?

The directory structure has stabilized and some FEPs now provide the context file (e.g. FEP-c390 context).

(Aside: I just updated the top-level post and the topic name with the new slug)

For FEP-888d I would like to request any comments on blockers, objections, etc. – for example, are there any mappings that one would expect to work, that don’t currently work?

The .htaccess is below for convenience (some changes as I test them):

Header set Access-Control-Allow-Origin *
Header set Access-Control-Allow-Headers DNT,X-Mx-ReqToken,Keep-Alive,User-Agent,X-Requested-With,If-Modified$
Options +FollowSymLinks
RewriteEngine on

# catch FEP-specific context requests
RewriteCond %{HTTP_ACCEPT} application/ld\+json
RewriteRule ^([A-Za-z0-9]+)\/?(.*?)?$ https://codeberg.org/fediverse/fep/src/branch/main/fep/$1/context.jsonld [R=302,L]

# catch FEP-specific context requests without content negotiation
RewriteRule ^([A-Za-z0-9]+).jsonld$ https://codeberg.org/fediverse/fep/src/branch/main/fep/$1/context.jsonld [R=302,L]

# catch FEP proposal documents
RewriteRule ^([A-Za-z0-9]+)\/?$ https://codeberg.org/fediverse/fep/src/branch/main/fep/$1/fep-$1.md [R=302,L]

# a generic catch-all rule
RewriteRule ^(.*)\/?$ https://codeberg.org/fediverse/fep/src/branch/main/fep/$1 [R=302,L]

I’m currently trying to put together some test cases in Python and have the following mappings:

{
  "fep": "http://localhost/fep/",  # this redirects twice basically, due to a quirk of apache
  "fep/": "https://codeberg.org/fediverse/fep/src/branch/main/fep/",
  "fep/xxxx": "https://codeberg.org/fediverse/fep/src/branch/main/fep/xxxx/fep-xxxx.md",
  "fep/xxxx/": "https://codeberg.org/fediverse/fep/src/branch/main/fep/xxxx/fep-xxxx.md",
  "fep/xxxx.jsonld": "https://codeberg.org/fediverse/fep/src/branch/main/fep/xxxx/context.jsonld",
  "fep/xxxx/asset.jpg": "https://codeberg.org/fediverse/fep/src/branch/main/fep/xxxx/asset.jpg"
}

This should support both IRIs of the form https://w3id.org/fep/xxxx/someTerm (to be defined in a plaintext file/asset, ideally) as well as IRIs of the form https://w3id.org/fep/xxxx#someTerm (to be defined ideally under a section of the FEP’s markdown file given an anchor of #someTerm). Furthermore, it supports importing FEPs singularly, as well as submitting an FEP that imports from other FEPs into its own context.jsonld

If there are no comments, changes, objections, etc. then I will update the FEP with the final .htaccess and request finalization. If/when the FEP goes FINAL, I will submit the PR to the w3id PURL service’s Github repo.

The intended way to define context.jsonld is explained in this FEP but if there are any clarifications or points of confusion, I can clarify those as well. See FEP-bad1 or FEP-a070 for examples

The option also remains open for anyone to use their own namespaces – see for example FEP-c648 which uses https://purl.archive.org/socialweb/blocked#blocked but could also potentially provide its own context.jsonld which would allow using https://w3id.org/fep/c648 to resolve and define like so:

{
  "@context": {
    "blocked": {
      "@id": "https://purl.archive.org/socialweb/blocked#blocked",
      "@type": "@id"
  }
}

The URL in this rule points to a HTML page. For example:

https://codeberg.org/fediverse/fep/src/branch/main/fep/bad1/context.jsonld

Shouldn’t it point to “raw” file instead?

https://codeberg.org/fediverse/fep/raw/branch/main/fep/bad1/context.jsonld

There’s a section called “Defining terms associated with an FEP”, but it doesn’t specify the file name (I assume FEP authors should always use context.jsonld ?)

The subsequent section “Example” still uses 9606 in context example and accompanying text.

The context example itself looks clear enough to me. I will update FEP-c390 context.jsonld accordingly.

exampleB is a term that links to another node

Is it acceptable to say “object” here instead of “node”? I think this will make it more accessible to people who are not familiar with JSON-LD.

#158 - WIP: Update FEP-888d - fep - Codeberg.org addresses the points you raised. Good catches!

PR still marked as draft because I am not 100% confident in my regex, so if anyone has a better grasp of regex than me, feel free to try to break them or otherwise simplify the .htaccess rewrite rules. If not, I will remove the WIP from it.

1 Like

Descriptiveness

I have an issue with the proposal related to descriptiveness, or rather the lack thereof. LinkedData is supposed to be both machine and human readable. Suppose on a msg I get a large @context and it references, say, 20+ different FEP’s. How do I read this context and know the meaning of these FEP’s?

Well, the FEP ID’s are semantically meaningless, so I must one by one follow the FEP namespace to find the corresponding spec and then see what they are about. Very unwieldy. In some cases property names in the namespace may be descriptive, but in many cases they probably aren’t. Consider e.g.

{
  "@context": 
    "order": "https://w3id.org/fep/14e3/order"
  }
}

What is meant by this, for example in the context of a hypothetical OpenRegulation app? Does it mean List Ordering, descending/ascending? Does it mean a Court Order ID? Or maybe an eCommerce Product ID for ordering resources for an upcoming case?

What if multiple definitions of the same property can occur in the msg? Which namespace prefix should one choose? One is free to choose anything, and there’s no convention to improve msg readibility.

{
  "@context": 
    "order": "https://w3id.org/fep/14e3/order",
    "myAppRocks:order": "https://w3id.org/fep/1a8c/order"
  }
}

Linked data has some well-known open standard ontologies where conventions for namespace prefixes are used (but not enforced) e.g. schema: or vcard: and ActivityStreams has a convention of as: prefix.

The full namespace of ActivityStreams itself is descriptive i.e. https://www.w3.org/ns/activitystreams → “Ah, the well-known W3C open standard :bulb:

Things get less clear on vendor-specific namespaces i.e. http://joinmastodon.org/ns# → “Ah, Mastodon-specific stuff :bulb:”, but “Oh, toot: is the namespace prefix to use for Mastodon properties, or is that standard? :thinking:

This non-descriptiveness is bound to become more of an issue as the list of FEP’s grows, and they cover numerous different domain models and functional aspects. FEP’s are like building blocks you stack together, but none of the blocks have an intuitive name that makes them easy to remember.

Fediverse Features

@helge started the Fediverse Features project that discerns concrete chunks of functionality, names them, and tests their behavior using BDD Gherkin scripts.

This project aligns with the idea that apps stack together functional building blocks:

:point_right: A FEP is a formal specification of a functional building block for the creation of interoperable apps.

:point_right: Interoperability on the Fediverse is not about apps, but on finding common denominators where compliance exists.

I find that second point major important, as this realization will help give developers the proper mindset to adopt when looking to integrate their project with the Fediverse. They should be focused on which common functionality they want to federate, and not whether arbitrary app A can interact with arbitrary app B.

The naming of Fediverse Features is something that can also help in making namespaces descriptive. For instance, taking the Groups FEP it may be indicated with its fediverse-feature name of groups:

{
  "@context": 
    "https://www.w3.org/ns/activitystreams",
    "https://w3id.org/fep/1b12/groups"
  }
}

Suppose my message supports all known group-types, so also Unbound groups FEP:

{
  "@context": 
    "https://www.w3.org/ns/activitystreams",
    "https://w3id.org/fep/1b12/groups",
    {
      "gs":"https://w3id.org/fep/2100/unbound-groups"
    }
  }
}

Note, in this example I use the namespace prefix suggested in the unbound groups FEP, but a better approach might be that besides namespaces following Fediverse Feature naming convention, they’d also have a preferred namespace prefix that is recommend accordingly, and that my be unbound: in this case.

i’m not sure @context was ever intended to impart meaning; it is simply a way to translate shorthand terms into full IRIs, as well as maybe specifying some extra metadata like @type: @id or @container: @list. basically, the only way you’d ever be able to tell fep/14e3/order from fep/1a8c/order is by reading the two feps. i don’t see a way around this, and it is an issue shared by other extension processes like XEP and PEP – there is no way to know what PEP 517 is about or XEP-0368 is about, unless you read those specs.


one hopes that a separate fep is defined that imports those 20 separate context files, but if 20+ different feps are provided, then 20+ different feps are provided. this and also

is why i think namespaces and compact term prefixes should generally be avoided where possible. this is a bit open-ended guidance, i realize, but it should be remembered and noted that namespace prefixes are something to be processed out before making any comparisons of IRIs. i need to play around with the json-ld playground a bit more before recommending anything concrete, though.

There’s a W3C Note: Best Practices for Publishing Linked Data that can be handy. In the Scope it says “Linked Data refers to a set of best practices for publishing and interlinking structured data for access by both humans and machines […]” and there’s like an optimum for both. The note also references TBL’s Cool URI’s Note.

One referenced PDF (broken linked, archived) shows a table summarising best-practices:

The last guideline is relevant in our context: Use a code if a local name may change. Which we propose to be the FEP’s code. But still all namespaces I have seen in the wild provide at least one word that allows a human to immediately classify it.

Not true. Consider if my namespaces were defined as follows:

{
  "@context": 
    {
      "lists": "https://w3id.org/fep/14e3/lists"
    },
    {
      "court": "https://w3id.org/fep/1a8c/open-court-regulation"
    }
  }
}

Then encountering a court:order would be meaningful and I immediately click the right FEP to check the specs.

Compare the following (random) lists and of some well-known namespaces and their common prefixes, with one of (hypothetical, in the future) well-known FEP’s and no prefixes defined:

And then these FEP’s…

Which has the best developer experience (DX), least frustration and chores?


Scope of the FEP?

This FEP is very useful for improving the FEP Process itself. It is also very fine-grained: It answers “How do we define FEP namespaces?” → Isn’t it too fine-grained and specific for its purpose?

The list of FEP’s is not a list of Architecture Decision Records (ADR’s). It is meant to help fedi developers to build interoperable solutions. The list of FEP’s is both short (we are just getting started) and already long (“Umm… where do I see ‘fep-888d’ in this list?”).

A better scope may be “FEP: Best Practices for ActivityPub Message @context” (likely a FEP that supercedes this one). This should be a separate discussion, but I wanted to mention here.

Update: A good input for applying best-practices is the Peertube @context I took notes on once.

I have since come to understand that this is the domain of OWL ontology files (at least, I think so).

Anyway, #158 - Update FEP-888d - fediverse/fep - Codeberg.org has been updated again and now contains guidance on how to format definition lists within an FEP.

1 Like

I think the next step here is to try to actually register /fep with W3ID and deploy the proposed .htaccess file there. If it works out, we can maybe try to get this FINAL?