FEP-888d: Using w3id.org/fep as a namespace for extension terms and for FEP documents

[tesaguri]

@trwnh

Off the top of my head, this seems to be what application/ld+json; profile="http://www.w3.org/ns/json-ld#context" is intended to solve… but I am unsure of how the wider JSON-LD ecosystem handles or fails to handle that profile parameter. It feels like potentially fragile behavior to say that the requester must specifically request the context profile and not just jsonld. But I could be wrong, this might actually be the correct intended solution.

Is your idea to return a context document when requested the http://www.w3.org/ns/json-ld#context profile and return an ontology document otherwise? If so, I think that works if the top-level @context of the ontology is a superset of the context document, since the JSON-LD Processing Algorithms set the loaded context value to the top-level @context value of the loaded document and ignores the rest of it when dereferencing remote context documents IIUC, so that it would only result in a slight waste of network bandwidth even when the client forgets to include the http://www.w3.org/ns/json-ld#context profile in the Accept request header.

But the situation might get subtler if the @context of the ontology has a term definition that’s not in the context document and that term overrides a term from another context document fetched by the client. To avoid such a conflict, the top-level @context of the ontology should be identical to that of the context document, but that means that you wouldn’t be able to use short-hand terms for ontology-specific vocabularies (at least at the top-level map), leading to less human-readable ontologies.

[trwnh]

@tesaguri

the JSON-LD Processing Algorithms set the loaded context value to the top-level @context value of the loaded document and ignores the rest of it when dereferencing remote context documents IIUC

I think this implies that you could serve a JSON-LD document with two keys, @context containing the intended context document for people who want to just to use the context definitions, and @graph containing the schema/ontology… but with the caveat that the @graph probably ends up having to use full IRIs. Which doesn’t seem like a real issue, considering that anyone intending to do schema/ontology stuff probably is LD-aware and knows what they’re getting themselves into.

For example, in an in-progress update to FEP-0499 I have the following contents included in a file fep-0499.jsonld which is a combination of context.jsonld as well as the RDF Schema converted into JSON-LD:

{
  "@context": {
    "multibox": {
      "@id": "https://w3id.org/fep/0499/multibox",
      "@type": "@id"
    }
  },
  "@graph": [
    {
      "@id": "https://w3id.org/fep/0499/multibox",
      "@type": "http://www.w3.org/1999/02/22-rdf-syntax-ns#Property",
      "http://www.w3.org/2000/01/rdf-schema#comment": {
        "@language": "en",
        "@value": "An optional endpoint used for wide delivery of activities to multiple inboxes."
      },
      "http://www.w3.org/2000/01/rdf-schema#isDefinedBy": {
        "@id": "https://w3id.org/fep/0499/multibox"
      },
      "http://www.w3.org/2000/01/rdf-schema#label": {
        "@language": "en",
        "@value": "multibox"
      },
      "http://www.w3.org/2000/01/rdf-schema#seeAlso": {
        "@id": "https://w3id.org/fep/0499"
      }
    }
  ]
}

And the other caveat is that if there are any terms used in your schema @graph that are also present in @context then those will end up compacted/term-mapped. But again, this should be fine and not cause any issues?

I think this would allow fetching like so

GET /fep/xxxx HTTP/1.1
Host: w3id.org
Accept: application/ld+json

No profile parameter required. Actually I think most JSON-LD processors seem to behave this way and will actually not provide the profile param when fetching, so this might end up being the safest approach.

The only thing I have to think about now is how to deal with existing FEPs that already define a context.jsonld and not a fep-xxxx.jsonld. This might end up necessitating some duplication where at minimum, fep-xxxx.jsonld MUST contain at least the exact same @context as context.jsonld

[trwnh]

One other thing I’ve been working on is that I wrote a Python script to auto-generate term definition files based on RDFa embedded directly in the FEP-xxxx.md file. I still need to figure out how to incorporate that into the general FEP workflow, and provide guidance around it.

it doesn’t limit itself to the standard library, is the challenge. it imports markdown, bs4, rdflib and json (and only the last of those is in the standard library i think?)

[codenamedmitri]

Oof, the Solid spec does this (embedded RDFa in the specs). It’s clever and labor saving in one sense… But the tradeoff is that the diffs (and the code itself) is way less readable as a result.

[trwnh]

It’s not a requirement to define your terms with RDFa markup, but I can’t think of a better alternative for anyone who cares about generating schema or ontology stuff. Parsing the Markdown is not feasible without mandating exact heading names, figuring out a way to somehow extract an entire section instead of a loose bundle of h2/h3 and p tags in the rendered Markdown, and a lot more that I haven’t even begun to consider.

If you don’t care about the RDF stuff then you can continue to define your terms in the Markdown without any additional markup. In that case I’d recommend following the existing guidance of having either a header or an HTML element with the given id and then using fragment identifiers for your terms. But I want to make it as easy as possible for people to do that extra work, and “authoring all metadata once” is a very big advantage to RDFa that I can’t overlook.

Regarding readability, it looks something like this for now:

<div id="multibox" resource="https://w3id.org/fep/0499/multibox" typeof="rdf:Property">
<h3>multibox</h3>
<dl>
<dt>URI</dt>
<dd><code>https://w3id.org/fep/0499/multibox</code></dd>
<dt>Label</dt>
<dd property="rdfs:label" lang="en">multibox endpoint</dd>
<dt>Comment</dt>
<dd property="rdfs:comment" lang="en">An optional endpoint used for wide delivery of activities to multiple inboxes. POST an Add activity where the object is at least one activity to be delivered, and the target is at least one inbox to deliver to.</dd>
<dt>Domain</dt>
<dd property="rdfs:domain" resource="https://w3id.org/fep/caca/EndpointMapping">EndpointMapping</dd>
<dt>Range</dt>
<dd property="rdfs:range" resource="rdfs:Resource">The endpoint (@id)</dd>
<dt>Functional</dt>
<dd property="owl:maxCardinality" content="1" datatype="xsd:nonNegativeInteger">Yes</dd>
<dt>Is defined by</dt>
<dd><a property="rdfs:isDefinedBy" href="https://w3id.org/fep/0499">FEP-0499</a></dd>
</dl>
</div>

EDIT: for more about this see this part of the FEP

Markdown specifications such as CommonMark currently do not support custom attributes, but some Markdown processors such as Goldmark support custom attributes on headers (but not on arbitrary elements). See [CM-ATTRS] for more discussion of this feature.

Consistent attribute syntax

If this ever got added (and there’s no timeline for adding it, unfortunately), then you could maybe do something like

## Terms defined

### multibox

<div resource="https://w3id.org/fep/0499/multibox" typeof="rdf:Property">

URI
: `https://w3id.org/fep/0499/multibox`

Label
: multibox endpoint
{property="rdfs:label" lang="en"}

Comment
: An optional endpoint used for wide delivery of activities to multiple inboxes. POST an Add activity where the object is at least one activity to be delivered, and the target is at least one inbox to deliver to.
{property="rdfs:comment" lang="en"}

Domain
: [EndpointMapping](https://w3id.org/fep/caca/EndpointMapping){property="rdfs:domain" }

Range
: The endpoint (`@id`)
{property="rdfs:range" resource="rdfs:Resource"}

Functional
: Yes
{property="owl:maxCardinality" content="1" datatype="xsd:nonNegativeInteger"}

Is defined by
: [FEP-0499](https://w3id.org/fep/0499){property="rdfs:isDefinedBy"}

</div>

This is of course non-standard and will not work. Also the div is still necessary because Markdown renderers generally don’t wrap sections in <section>, which would probably be preferable to a plain old div, or as Markdown defaults to, nothing at all whatsoever.

[trwnh]

@trwnh

The only thing I have to think about now is how to deal with existing FEPs that already define a context.jsonld and not a fep-xxxx.jsonld. This might end up necessitating some duplication where at minimum, fep-xxxx.jsonld MUST contain at least the exact same @context as context.jsonld

[888d] Support machine-readable term definitions, URL hacking, and alternate...

I have concluded that the duplication is indeed necessary for now. Part of this PR involves copying all existing context.jsonld to fep-xxxx.jsonld in preparation for a PR to perma-id/w3id.org on github. After that PR is made and merged, JSON-LD conneg will no longer redirect to context.jsonld, but instead will redirect to fep-xxxx.jsonld.

[trwnh]

One additional issue that I just discovered is that Codeberg’s Markdown rendering seems to have introduced issues at some point.

• The “algorithm” section for the Python script now renders as a broken ordered list. I’m not sure how to fix it (since I took efforts to explicitly NOT have it parsed as an ordered list at the time), but I think I can figure something out and do a PR for that… EDIT: PR done
• More crucially, explicitly defined id attributes are prefixed with user-content- which breaks anchors in the rendered HTML. This isn’t a problem for generated RDF (fep-888d.ttl, fep-888d.rdf, fep-888d.jsonld), but it is a problem for the HTML rendered for fep-888d.md. Links of the form https://w3id.org/fep/888d#SomeType would not take you to the correct fragment; links of the form https://w3id.org/fep/888d/SomeType continue to work. I don’t know what to do here other than maybe file an issue against Codeberg? EDIT: issue filed, apparently they have Javascript that intercepts the navigation so it’s not entirely broken, but it is broken for anyone parsing the HTML or disabling Javascript.

[silverpill]

It seems that some JSON-LD libraries don’t accept FEP-888d contexts because Codeberg serves them with text/plain content type.

[eprodrom]

I posted an issue here:

Move context documents off Codeberg Pages

I’ll repeat the content verbatim:

I have very inconsistent support for getting context documents for FEPs. I get about 50% or more failures from the platform, with slow responses. I attached the output from the ab program to show the low reliability.

The status page shows 66% uptime:

https://status.codeberg.eu/status/codeberg

…and the main page for codeberg pages has a banner saying not to use it for production sites:

https://codeberg.page

Since most FEPs use the fediverse/fep: Fediverse Enhancement Proposals - Codeberg.org PURL namespace from FEP 888d, it should be possible to deploy the files to another static hosting service without changing clients; just change the redirect at w3id.

There are other static site hosting systems, like archive.org’s S3 service:

https://archive.org/developers/ias3.html

Also, companies like Fastly have been supportive of the Fediverse, and might be willing to give credits for static site hosting.

https://www.fastly.com/fast-forward

[eprodrom]

Can we please federate this topic so I can participate remotely?

[aschrijver]

Via the Codeberg crew I learned that the Verge had published an article that links to the Mastodon quote posts specification.

[Mastodon] also worked on a technical specification for quote posting in the fediverse and hopes that other platforms will adopt it.

Problem is that this uses the full URL to the markdown doc, and not the short-hand w3id.org URL. The more direct URL’s are used, the more broken links there will be should for some reason or other the FEP change its location some day.

DON’T DO: https://codeberg.org/fediverse/fep/src/branch/main/fep/044f/fep-044f.md
DO: https://w3id.org/fep/044f

This best-practice of linking may need more attention and promotion, so it becomes the norm. Additional advantage is that - where acknowledged news media link to FEP specs - w3id.org/fep article references allows “Fediverse enhancement proposals” to get its own Wikipedia page.

I will see if I can reach The Verge for a correction.

[helge]

@aschrijver

DON’T DO: https://codeberg.org/fediverse/fep/src/branch/main/fep/044f/fep-044f.md
DO: Making sure you're not a bot!

This best-practice of linking may need more attention and promotion, so it becomes the norm. Additional advantage is that - where acknowledged news media link to FEP specs - w3id.org/fep article references allows “Fediverse enhancement proposals” to get its own Wikipedia page.

This CANNOT be the official FEP policy as this topic is not final. Also it would be good to separate out the issue of Links to the Markdown files to all the JSON-LD stuff.

If we made these links official policy, one can also use them in an eventual static site: #673 - WIP: Create a FEP static site - fediverse/fep - Codeberg.org

[aschrijver]

But nowhere in the FEP process does the document get deleted. w3id links are valid as soon as a FEP document becomes part of the collection, and that should not be tied to its status imho.

Update: But I will not ping The Verge with unclarity on what is best practice.

[aschrijver]

Another official news source is linking to the FEP, and that might be good enough to have a Wikipedia page for the FEP perhaps. Perhaps not when it links to codeberg.

We have..

• Mastodon is bringing quote posts to the fediverse | The Verge
• Mastodon rolls out quote posts with protections to prevent 'dunking' | TechCrunch

[trwnh]

i think it could be brought to FINAL if there are no objections remaining. of course being FINAL doesn’t mean the .htaccess redirects can’t be updated in the future; the scope of this proposal is simply to have the redirects be managed by w3id.

in a practical sense we have some feedback from @eprodrom that codeberg’s reliability has been somewhat low. if there are better destinations to host a copy of the repo, now would be the time to suggest them. a deploy hook could be configured on the fep repo to deploy fep resources to that host.

i don’t think there needs to be any separation between formats; rather, each fep gets its own folder to include whichever resources it wants to include. at minimum, a fep-xxxx.md can be rendered into an index.html, and most likely a jsonld file that can be used as a context document (which fep authors can locate anywhere within the fep folder, but the two approaches that make the most sense are to either call it something like context.jsonld or to otherwise use the fep-xxxx.jsonld to serve both a @context and a @graph in the same document)

[silverpill]

@trwnh

i think it could be brought to FINAL if there are no objections remaining.

There are two outstanding issues:

FEP-888d: Using w3id.org/fep as a namespace for extension terms and for FEP documents

It seems that some JSON-LD libraries don’t accept FEP-888d contexts because Codeberg serves them with text/plain content type.

FEP-888d: Using w3id.org/fep as a namespace for extension terms and for FEP documents

I posted an issue here: I’ll repeat the content verbatim: I have very inconsistent support for getting context documents for FEPs. I get about 50% or more failures from the platform, with slow responses. I attached the output from the ab program to show the low reliability. The status page shows 66% uptime: https://status.codeberg.eu/status/codeberg …and the main page for codeberg pages has a banner saying not to use it for production sites: https://codeberg.page Since most FEPs use the…

Though personally I won’t object to finalization if these issues are not resolved.

[silverpill]

@helge

This CANNOT be the official FEP policy as this topic is not final. Also it would be good to separate out the issue of Links to the Markdown files to all the JSON-LD stuff.

+1 on both points

I don’t think that using a 3rd party link shortener service is a good idea. The FEP site is in development, and people will be able to link to it once it’s ready.

[eprodrom]

So, I’ve got another question: could we get some implementation reports here? I mentioned before my problems with Codeberg Pages reliability and performance.

I also think there’s a problem with using the context documents. If the client uses Accept: application/ld+json the response is correct:

evanp@MacBookPro activitypub-inverse-properties % curl -L -H "Accept: application/ld+json" https://w3id.org/fep/5711
{
  "@context": {
    "inv": "https://w3id.org/fep/5711#",
    "likesOf": {
      "@id": "inv:likesOf",
      "@type": "@id"
    },
   // ...

However, requesting application/json will return the Codeberg page for the Markdown document:

evanp@MacBookPro activitypub-inverse-properties % curl -L -H "Accept: application/json" https://w3id.org/fep/5711       
<!DOCTYPE html>
<html lang="en-US" data-theme="codeberg-auto">
<head>
	<meta name="viewport" content="width=device-width, initial-scale=1">
	
	<title>fep/fep/5711/fep-5711.md at main - fediverse/fep - Codeberg.org</title>

I think the JSON-LD should be returned in this case.

[helge]

Probably once we are finished with the static site, something like https://fediverse.codeberg.page/fep/a4ed will become the standard.

WIP: Create a FEP static site

We probably should stick with the repo links until the new standard is ready ….

[aschrijver]

@helge

Probably once we are finished with the static site, something like https://fediverse.codeberg.page/fep/a4ed will become the standard.

Since codeberg.page is in maintenance mode and stated to be not suitable for production-use, that is not the best domain imho. Another option, relating to proposal to have a portal at activitypub.eu, is to set up subdomains of socialhub.activitypub.eu and fep.activitypub.eu in DNS. See also: Proposal: Run SocialHub forum at activitypub.eu core domain