How would using multiple vocabularies look like with ActivityPub


I’m trying to understand how ActivityPub works with different models. If I want to make use of the model in Web Annotation Data Model, to build an activitypub server, how would that work? I see there is an activitystreams vocabulary Activity Vocabulary, but that doesn’t seem to be defining annotations that I would like to use. Is there a way to merge these?

The general answer is that they can be merged with a properly designed JSON-LD context. However, you haven’t shared much about what you want to do, so the specifics could vary. For example, do you just want to store annotation information referring to plain AS objects or do you want to integrate annotations into the AS objects somehow? The latter will require an extended JSON-LD context. The former might not.

1 Like

Essentially I want to build an annotation server that is federated. Basically, a social federated annotation server. Someone should be able to annotate a website, annotations get sent to the server and to followers, and followers can then go in and view those annotations.

In that case you should define an object type for an Annotation. The best way forward is ideally to work with an existing linked data format for annotations to use for your AP object, and adapt the @context accordingly. And then, write a Fediverse Enhancement Proposal, or fep, and ask feedback from the community to improve further on your specification. You can see existing FEP’s at of which the discussions take place on this forum.

Update: In this toot someone mentions working on Linked Data image annotation, just FYI.

What are your interoperability goals? Will the annotation servers only federate each other (at least, initially) or do you also want to federate with microblogging platforms like Mastodon? Federating the annotation data will be awkward, at best, with the microblogging platforms. You’d probably need to need transform the annotation objects into formatted AP Note objects of some kind.

How will you be processing the data? Will you be using plain JSON parsing and interpretation or using JSON-LD/RDF processing (both are options with JSON-LD representation)? I’m asking because there’s a minor issue with the terms in the Web Annotations and ActivityPub contexts overlapping ( target, for example). JSON-LD processing would ensure the JSON keys are fully disambiguated. However, it may also be possible to hard-code which term is being used in a specific situation .

Those are some challenges, but there’s good news too. The Web Annotations context uses ActivityPub terms for some purposes. It looks like the collections and paging representations are using AP terms, for example.

A Create activity might look like:

    "@context": "",
    "type": "Create",
    "id": "https://social.example/alyssa/activity/a29a6843-9feb-4c74-a7f7-081b9c9201d3",
    "to": [
    "actor": "https://social.example/alyssa/",
    "object": {
        "@context": "",
        "id": "http://server.example/anno-123",
        "type": "Annotation",
        "body": "http://server.example/my-annotation",
        "target": "http://server.example/blog-article"

Some people will say you need to add Object (will expand to “ActivityStreams 2.0 Terms”), in the annotation type property, but it’s not required and doesn’t appear to serve any purpose in this case.

You can expose the annotation collections using the streams property in the actor profile. However, there are also other options for exposing the collections along with some metadata.

Depending on how you manage the annotation collections, you may want to consider Add and Remove activities instead of Create and Delete.

When publishing, you could theoretically know which actors are using various AP implementations (via nodeinfo, which is supported by most platforms) and then format the outgoing activity to be compatible (for read-only notification interoperability). For Mastodon-like platforms, maybe the Annotation would be converted to a Note with the annotation body as the content with an embedded link to the target. Currently, Mastodon would ignore an Annotation-typed object.

Given how mixed the support is for json-ld processing among fedi software, I would suggest putting all the vocabularies you intend to use in the root @context object. I think that’s going to be your best shot at making them useful to arbitrary recipients.

I can see a few minor potential advantages for the object-level context for producers (since that’s where it is applicable in this case). However, I don’t see it making any difference for recipients. The only implementation I’m aware of that will do anything with an oa:Annotation object type is Vocata. It uses a compliant JSON-LD processing library so it would have no problem with either context location.

1 Like

Thanks for the pointer to the Web Annotation specification. I’ve been working on porting my fediverse bookmarks collection to an RDF graph store. I had been modeling bookmarks as AS2 Link objects but the Web Annotation model is much better. After the standalone bookmark server is working, I’ll create a public code repository and investigate adding AP federation (based on oa:Annotation objects). If you release something in the meantime, please let me know. Thanks again.

1 Like