Retooling ActivityPub servers for richer clients

Hey SocialHub! :hugs:

This is my first post here. I’ve been exploring Fedi for the better part of two years, but I’ve only recently become interested in properly understanding the spec.

I have a little shell alias to make ActivityPub requests with curl, and I sometimes use it on random links out of curiosity. And doing so has made me curious about the potential of ActivityStreams objects in a C2S context.

If you exposed a client to an ActivityStreams object, I think it’d be pretty trivial to write a client that can render any kind of object. Right?

The biggest problem with Fedi imo is the lack of cohesion. Mastodon is strictly written to handle microblogging, and so that’s all its database will store and all its API is written for. As a consequence of Mastodon being so huge, microblogging has basically become Fedi’s main schtick. Anything outside of that requires a separate ecosystem.

Case in point, when viewing a PeerTube video from Microfedi (microblogging Fedi), it’s just a link to view the video on the original website. But the ActivityStreams object of a PeerTube video contains all the information a client would need to view the video natively. So, why should I need a separate app to view a PeerTube video?

I think ActivityPub implementations are limiting this kind of progress by ignoring C2S. Especially when ActivityStreams is so rich with context that can used by those clients.

We’re also in danger of EEE if corpos create this wholistic version of the Fediverse first.

Federating recipes :cookie:

Imagine I wanted to define a Recipe object.

The main purpose of a Recipe object would be to list ingredients explicitly, with a standardized set of names for each measurement. This provides additional context for a client to do some interesting things. Like maybe we want to convert the measurements into metric, or add an ingredient we don’t have to our shopping list.

Under the current paradigm, this object can only be displayed in the context of a “Federated recipe platform!” :frowning:

The object might look something like this:

    "@context": [
        "https://www.w3.org/ns/activitystreams",
        "https://w3id.org/security/v1"
    ],
    "id": "https://example.com/recipes/8420582323800",
    "type": ["Recipe", "Article"],
    "name": "Macadamia Nut Cookies",
    "summary": "<p>I found this in my grandma's old cook book!</p>",
    "content": "<p>In a large mixing bowl, whisk together the flour, baking soda,..</p>.",
    "attributedTo": "https://example.com/actor/cat",
    "to": [
        "https://www.w3.org/ns/activitystreams#Public"
    ],
    "cc": [
        "https://example.com/actor/cat/followers"
    ],
    
    "attachment": [
        {
            "type": "Image",
            "content": "The cookies in question 😋",
            "url": "https://example.org/static/53489992380692642.jpeg"
        },
        {
            "type": "Ingredient",
            "name": "All-purpose flour",
            "units": "cups",
            "measurement": 2.5
        },
        {
            "type": "Ingredient",
            "name": "Baking soda",
            "units": "tsp",
            "measurement": 1
        }
        ...
    ]
}

Alright, so let’s say I bite the bullet and make a Fediverse recipe platform :clap:

A blog server like Plume might fetch this and recognize it as an article; but the reader can’t view the ingredients, and so they can’t make the recipe!

Ideally, ActivityStreams definitions should contain as much context as reasonable, while also re-using keywords in places where it makes sense to. However, an ingredient really must be expressed and interpretted as an ingredient for this to work.

One alternative is to define an Ingredient additionally as a similar object. Or, you could accept that there will always be some unfederated data. To me, the ideal solution is to offload the rendering of this information to the client.

Retooling ActivityPub could solve problems with the rest of the internet

I’ve spent some time exploring Geminispace, and listening to people complain about the current shape of the web. (Daily reminder that Chromium has 35 million lines of code.) Also, try looking actually looking for a recipe online. Ads galore.

But something about the Gemtext solution of “just simplify the markup” feels wrong.

I think I’ve settled on the opinion that building the internet around a markup language was a bad idea. Sure, it’s cool to zip around from site to site, knowing you’re seeing it as the author intended. However, there are really important things that markup cannot express. This is evidenced by the fact that every service under the sun feels the need to have their own special API.

I think the web should be built around a vocabulary that can express action, intent, and context - like ActivityStreams - instead.

Conclusion

Thanks for reading. I really wanna know what this community thinks about this perspective :heart:

4 Likes

Welcome to the community @Catboomer :wave: and you are spot on in those observations. On one hand we are quite happy about the growth in numbers and developer interest caused by The Muskening™. OTOH there are fears for corporate capture, upto EEE and changing cultures (Eternal September).

If you didn’t read it yet, you might like the take in Ideating organization structure for the Grassroots Fediverse (wiki) to organize in two parallel tracks, a grassroots one and a formal one. This forum is like a DoOcracy where anyone can take initiative to bring improvements.

There’s quite a bit of interest in the C2S parts of the specs these days, and - related to that - fill in the missing bits and pieces to implement solutions that with a decent level of interoperability. The extensibility mechanism of AS/AP needs to be improved / formalized, and there’s some talk about Capability Discovery and Compliance Profiles whereby servers and clients can figure out what’s supported by particular actor endpoints.

You can help by participating in the Fediverse Enhancement Proposal process. See existing FEP’s. Just yesterday a new FEP was added, proposing an extension for Valueflows. For recipes something similar might be done.

1 Like

Yes, that could work. Bear in mind that the AP spec has quite a few flaws and bugs, that may need to be fixed

For example, this is not there, sadly :frowning:

Rendering each object on client side based on type is a valid technique. In fact, that’s the exact approach favoured by Tim Berners-Lee in his SolidOS project.

What you could do is make a recipe type and recipe vocab then link that from your object

As clients get smarter more get the ability to render it. Though in these client type solutions there is a degree of authors prioritizing the low hanging fruit. So recipes as a use case would need to have enough mind share to make clients want to change the code.

Good post though, good thoughts, welcome to socialhub!

1 Like

Welcome to SocialHub Catboomer! :partying_face:

Totally! :stuck_out_tongue:

sc;nr I mean it’s trivial in a language such as python

from rich import print
from bovine import BovineClient
print(await BovineClient.from_file("helge.toml").proxy_element(args[1]))

This is quite literally how I would do it nowadays with the bovine python library.

I actually think this would be one of the best case scenarios. If corporations were to push a way to create a FediVerse with support for all kinds of contents instead of applications designed for a specific purpose.

Personally, I think it just takes the right mindset to design stuff that can interact with any kind of content. I wrote about how to do it here. As long as we are in the case that people provide a “fallback webpage” for there content, the FediVerse remains usable.

Unfortunately, nobody has written the FediVerse Application with nice failure behavior yet. Doing so is outside my skillset.

1 Like

Thanks for the warm welcome everyone! :tada:

The recipe object was more of an example. :sweat_smile: I’m more interested in progressing C2S at the moment.

I believe that the ideal situation for a general purpose ActivityPub client would be to support W3C-approved objects out of the box, but provide an easy way to write and install plugins.

As in the Recipe example I gave, a Recipe is not in the ActivityStreams standard - however an Article is. So when viewing a Recipe, the app can prompt the user to search for a plugin that implements the Recipe object. If they don’t have internet access or aren’t interested, they could simply view it as an Article instead.

Interesting! I had no idea that Bovine was approaching this problem. I’ll take a closer look at it.

1 Like

Recipies example

What you can do is put the ingredients in the content again as a fallback mechanism.

A similar approach is used in FEP-e232. In that FEP the fallback content is “easy” for everyone. AP implementations that don’t understand the FEP will often render a link preview and other AP implementations know what the Micosyntax is and thus also know how to remove it again if they have a better way of displaying it. Maybe a general mechanism for fallback content would be necessary, perhaps also something for consideration for the Capability Discovery that @aschrijver mentioned.

C2S in general

As for C2S support I think the issue is that “microfedi” and surely also other AP implementations due to their specialization have a set database structure and are not able to store arbitrary AP objects in their database. They take only some attributes which they are interested in and throw away everything else. So unless server implementations start storing arbitrary JSON in the database the C2S support will probably stay limited to the same set of attributes/types.

2 Likes
  1. ActivityPub does not transport arbitrary JSON, but JSON-LD objects. This makes a big difference. In particular, the objects one will store will not have arbitrary keys, but keys with well defined semantic properties.

  2. You are actually not forced to store anything except the reference to the retrieved object. You should be able to look everything else up.

  3. People who are interested in how one can implement a database to store json-ld content, take a look at bovine_store.

1 Like

I honestly don’t believe in the general purpose ActivityPub Client. To use a metaphor, there will always be the tool you cannot add to your Swiss Army knife. For example, a shovel.

What one however can do is require all object types to provide properties to use as a fallback, for example the object

{
   "@context": "...",
   "type": "MagicalThing",
   "summary": "This object is pure magic; it is beyond your comprehension",
   "url": "https://magic.school/some_uuid",
   "spellForm": "6tX07me4%%B",
   "handGestures": ".....",
   ....
}

might be incomprehensible to your Client, but it should be clear how to display a fallback, just use Mastodon’s algorithm for Article’s. Similarly, I’m unsure how much more of a plugin mechanism than “Show the url in a webbrowser”, one really needs to start with.

In summary, in my eyes the way towards general objects seems pretty well defined. We are just missing the applications that actually use them. Adding “Share with your ActivityPub Client” to umai might be one of the better ideas.

2 Likes

I should clarify I was talking about the C2S part as in “sending an activity”, i.e. client to home server. For that of course the server which is publishing will need to store arbitrary JSON-LD to then deliver or serve it, instead of just storing a reference for most use cases.

I said JSON because syntactically they are the same and while e.g. PostgreSQL has a JSON data type there is no JSON-LD data type. I.e. anything where you can store “arbitrary JSON” you can also store “arbitrary JSON-LD”. With arbitrary I mean being able to control any (arbitrary) key and corresponding value in the resulting representation as opposed to somewhat controlling specific keys.

Since ActivityPub is essentially RDF, a storage format may also be a triple store.

I think we are misunderstanding each other somewhere. I’m just not sure how. I’ll to try summarize my thinking

  • I’ll use ActivityPub Actor Host as explained here to talk about “ActivityPub Servers” having the capability to handle arbitrary Activities and Objects
  • I agree building one of these servers, one wants to store content as JSON.
  • I disagree with calling it arbitrary. At least in my eyes, the json documents one is storing have structure. They can be identified by an id. They have a type. Depending on the type, more requirements should exist.
  • With appropriate definitions, one should be able to route Activities from the inbox to appropriate Clients. Nobody has build something like this yet, because the FediVerse hasn’t made it necessary yet. It will probably happen if something like e2ee is implemented.