openEngiadina: From ActivityPub to XMPP

@pukkamustard
Ok, I see activityPub servers as messaging systems and the following 3 things in ActivityPub:

  1. transport protocol (http)
  2. syntactic message (json-ld)
  3. semantic message (e.g. create activity)

If we would distinguish these 3 things, then the semantic message could be transferred via xmpp and still have activitypub compatibility.
Of course I see difficulties in translating different syntactic mappings, but not if we say the messages are RDF instead of json-ld.
Then it is conceivable to offer translators or even bridges as a service. a bridge would also be conceivable for the transport protocol.

It would be nice if we can distinguish these 3 levels at activitypub kombatibilität. then someone could use xmpp and still be compatible.

btw. i’m not sure if there is already an activitypub implementation that really speaks json-ld. But I haven’t looked at the S2S interfaces yet.

Are there xmpp clients that understand rdf/xml? and which semantic messages would you transfer ?

which transport protocol does xmpp use?

1 Like

we also need a meta data request, which tells a client what the server supports. e.g. which activities are supported, which object types/attributes are supported, C2S, S2S, …
maybe which transport protocol, syntactic, ?

Agree! It would be very nice to make content (the semantic meaning) be independent of particular serializations and transport protocols. Then it really does not matter so much if content is transported over HTTP, ActivityPub, XMPP, some peer-to-peer network, a USB stick or pigeons.

One important thing is not to make identifiers of content tied to protocols. The solution to that: content-addressing (a la Datashards and ERIS).

I’m not aware of any XMPP clients that understand rdf/xml. And I think I would transfer ActivityStreams activities.

The default is TCP (defined in RFC 6120) but there are others (WebSocket, HTTP and some people seem to be using XMPP over high-latency radio links).

In XMPP there are extension that do this. For example: XEP-0030: Service Discovery and XEP-0156: Discovering Alternative XMPP Connection Methods.

My intentions are not to convince anybody to use XMPP (it has it’s fair share of issues). I just wanted to explain why I believe XMPP is currently a more pragmatic solution for the very specific needs of openEngiadina.

I do want to encourage decoupling content from protocols with ideas like content-addressing.

I still waiting for a content-addressing java-lib :wink:

I am thinking about addressing my 2nd AP server via AMQP. And then offer HTTP bridges. Let’s see, still have to play a bit.

A couple of aspects to this

The first is coupling data to the domain. So in general in Solid we tried to avoid this by having profile items such as users as relative links to the page e.g. #me. Most other systems dont do that, creating a coupling to the domain. I think fediverse and web2.0 have areas to develop in this regard. The good news is that the fix is easy in JSON-LD, you just add “@id” : “#me” to the user object, if anyone wants to go down the slightly more decentralized path

In such a way then objects in a web page become like JSON-LD objects that could be sent over any protocol, reasonably easily. They could be translated to XML too as there’s converters around. So you have a User object, a Message object. I once built a p2p browser to browser chat app using peerjs that did exactly this, and it seemed to work

The other apsect coupling to the protocol, ie by using http vs another URI scheme, such as a content addressible URI scheme. For example, the ni: “naming things with hashes scheme”. Or I guess the did: scheme, tho I’ll point out that it’s not all that decentralized. For example it has a hard dependency on w3.org (an http uri), the very thing it would aim to replace

I’ve settled here on a hybrid solution and that is to have a content addressible timestamped Uri in an http page, that can change over time, and you change the time stamp. So you can start of in one place, and then you record the history of where things go as they move. Anyone that respects your timestamping system will then be invited to update their links, possibly with the help of redirects in the mean time. Doesnt help if someone write down a URL on a napkin, but a move in the right direction

tl;dr clean up data models and they can be sent over any protocol xmpp, http etc & content addressible IMO works better WITH http, rather than, in competition with http

1 Like

Somehow frustrating.
Did you have any concrete problems with the expanding algorithm?

PS wrote in the fediverse cause it federates …

A very concrete problem was that the Framing Algorithm is not yet implemented in the JSON-LD library we are using in CPub (GitHub - rdf-elixir/jsonld-ex: An implementation of JSON-LD for Elixir). This was a blocker for implementing C2S (and eventually S2S) as we could not serialize JSON-LD in the form that ActivityPub implementations expect.

Another problem is that there are very few implementations and it is imho very hard to implement the JSON-LD algorithms so that they are compliant. There are implementations for some popular languages (JavaScript, Python and more), but if you choose a slightly less popular language, you’re out of luck and JSON-LD becomes a huge challenge that you need to face. For openEngiadina we were and are using not so popular languages and this makes JSON-LD a real problem.

One might say: then just use popular languages for which there are JSON-LD implementations. I disagree and prefer not having to use a specific platform/language just to be interoperable.

2 Likes

I see.
So the problem is the way in which current implementations expect things.
We will never be able to solve a henn-and-egg problem then.

I would wish that I know anything about elixir and prefer to have these 800 lines
in elixir too jsonld.js/frame.js at 20e2286d198ce7d376320306f9df3667f48a4544 · digitalbazaar/jsonld.js · GitHub
Maybe Manu knows [or digitalbazaar has] elixir people who could contribute it to the mentioned library (?)

the challenge will not only be to have json-ld parsers/generators in different languages. the bigger challenge will probably be to make them compatible :wink:
I am very, very curious. I will use existing json-ld libs in my implementation and will report what that leads to. Since the day before yesterday i’m hacking again, let’s see how long i can keep it up. but i’m working inside with rdf4j as abstraction and transport turtle. json-ld will “only” become an in/out adapter/translator in my case.

Once I’m a little further along, I’m happy to test json-ld compatibility with others. i’m starting with C2S though.

1 Like

I worked with JSON-LD parsers for a long time

But the large footprint and buggy implementations have lead me to favour a different approach

And that is to use general native JSON parsers, and extract the relevant LD from the payload

It is a very nice, encouraging article.
One more important point is

“And why has everything become an array?”

Cause it does not help anybody if software expects it not to be an array when ActivityPub clearly says, it can be an array.

The Fediverse needs JSON-LD alone for not loosing content.
Have a look at e.g. Federation - Mobilizon – you would loose all the schema.org properties when you just use the JSON payload.
If you don’t use JSON-LD, your software will not know where the event is and all users are lost. This is one of 1000 examples.


about

“There are practically no implementations of the ActivityPub Client-to-Server protocol (C2S)”

I am leading a team at a large publishing house writing a server and a client and I am using every free minute to write my Open Source client, see Commits · redaktor/widgets-preview · GitHub

And so the above sentence is just frustrating and disencouraging to read that there are no implementations (apart from:)

  • pleroma,
  • and-status
  • smithereen (see sloph to smithereen demo)
  • independent clients and more
1 Like

Thank you for pointing out the existing C2S implementations. I changed the wording in the post to “There are not many implementations of the ActivityPub Client-to-Server protocol (C2S).” and linked to your post.

It was not my intention to frustrate or discourage. Sorry that you felt so.

Well, it is one choice less for our users now … And you were the only ActivityPub project who made it through that strange NGI0 thing, I think.

Wondering how other ActivityPub Elixir devs will handle it, e.g. https://lemmy.ml/post/95684

Anyway: What should we do with the Talk then?
It is false information now…

Absolutely not ! json-ld is still linked data. Even it’s awful :wink:
My implementation will understand json-ld and it will also generate json-ld. I’m just afraid that very few people can interpret my json-ld. I currently working on an docker image.
Features now:

  • Receiving a create activity
  • read activities/Objects by their subject
  • no authenticatiopn yet
  • no actor profile yet.
  • supports json-ld, turtle and many other rdf syntax.

maybe we should think about a webservice that translates json-ld to turtle and vice versa.:wink:
I have one processor each for (in/out) that does this translation for me

What I meant is that the info with ActivityPub in the video is not up to date and viewers could be thankful for an update (maybe via comment) …

that translates json-ld to turtle

Yes maybe – will look what turtle is is exactly.
I am using the “reference implementation” of the JSON-lD tools for node.js …

Manu Sporny was also engaged at the Conference and it works nicely for me.

Sound’s great, so your client can be the first test client for my C2S Server :wink:

1 Like

Well, just some coming to my mind …
There is much more !

This post is a wiki, add yours:

Implementations of the ActivityPub Client-to-Server protocol (C2S)

Server

Also leaving this here

PS: Not sure about https://bonfirenetworks.org but Mayel is also doing Elixir, see GitHub - bonfire-networks/activity_pub: Modular ActivityPub library in Elixir

1 Like

Some more description by @pukkamustard on rationale of using XMPP as discussed on Matrix:

Do you think I should go for XMPP? What would be the pros and cons?

I think you should go for XMPP. There are three main advantages I see:

  1. Everything you need to get it working is well specified. You need to read a RFC and a couple of XEPs. Unfortunately ActivityPub seems to be underspecified and getting a working/federating implementation is non-trivial.
  2. There are working, well-tested and generic XMPP servers that you can just use. Generic in the sense that XMPP servers do not care what kind of content you transport over them. This is not the case for ActivityPub. I am not aware of any ActivityPub server that allows arbirtary types of content to be transported. This was something we were trying to do with CPub. With XMPP we just use ejabberd or prosody (both work perfectly fine).
  3. With XMPP you’re not bound to Web technologies. XMPP main transport is TCP, but it works over WebSocket (what we are using in GeoPub) and other things such as radio links (XEP-0361: Zero Handshake Server to Server Protocol). ActivityPub is a Web first stack. If you create a non-web client you still need a lot of web libraries.

Cons:

  • You need to read some XEPs and other documents that look scary.
  • You need to deal with XML. But IMHO that’s better than having to deal with JSON-LD.

Can we easily bridge XMPP with ActivityPub?

Libervia project is working on this (NLnet; XMPP-ActivityPub gateway).

GeoPub uses ActivityStreams over XMPP. Which should make a bridge very easy (ActivityStreams over XMPP — inqlab)

Later on @pukkamustard mentions:

Of course there are a couple of generic ActivityPub servers in development. Among them rdf-pub by @naturzukunft (linkedopenactors / rdf-pub · GitLab)

Also worth mentioning ActivityPods (GitHub - assemblee-virtuelle/activitypods: ActivityPub + Solid PODS = ❤️) and Bonfire (https://bonfirenetworks.org/)

(There’s also https://semapps.org)

Considering that many people, who are doing generic servers, looked for help –
and considering all the meetings, I disagree.
Also, I started GitHub - redaktor/server which just went the closed-source-way for G+J cause we did not get funding at nlnet …
It seems to be a vicious circle but maybe we should actively collect the efforts.

Apart from go-fed, python ap, epicyon, bonfire, semapps … there is also a Rust start now: