Fediverse Relays

ActivityPub
6

Here’s a blog post I wrote almost a month ago trying to explain (Mastodon-flavored) relays. It’s in the queue for the work blog, but in the meantime I’m posting it here in case it’s useful to anyone.

Notes on talking to Fediverse relays

by Darius Kazemi
Sep 10 2024

I was recently trying to write some software that can subscribe to a Fediverse/ActivityPub relay, and I was surprised to find that there’s very little documentation about relays online. Usually my go-to for niche ActivityPub implementation stuff is the SocialHub Fediverse Developer Portal, but I couldn’t find much there. I ended up having to dive into a bunch of source code and puzzle some things out from various READMEs and don’t want anyone to have to do that again!

But before I get into the tech, let’s talk a bit about relays and why we might want to talk to one.

What is a Fediverse relay?

A relay is a piece of software designed to solve a Fediverse-specific problem, so I’ll describe the problem first.

The problem: it’s hard to crawl a network

Let’s say I’m starting a new Mastodon server. (I’m using Mastodon here but the general principles apply across other software too.) I put this new server online at coolsite.example. That server doesn’t actually have any way of knowing that mastodon.social or any of the ~20,000 other servers out there exist. Normally the way a server learns about other servers is through user action. In the case of coolsite.example it just came online, and as such I am the only user with an account at @darius@coolsite.example. Just lil’ ol’ me on an empty server with no messages to read.

That is, until I type @elonjet@mastodon.social into my search bar. What happens then is (roughly and simplified):

  • coolsite.example contacts mastodon.social and asks “is there an account on your server called elonjet?”
  • mastodon.social sends a message back to coolsite.example saying “yes there is, and here is where to find it”
  • coolsite.example shows me the profile for elonjet
  • If I press “Follow”, it sends a follow request to the inbox for @elonjet@mastodon.social, which automatically sends back an approval. The approval is a promise from mastodon.social that it has put @darius@coolsite.example into a list of accounts that should be sent a copy of messages that @elonjet@mastodon.social posts
    • Being on this list makes my account a “follower”! From now on, when @elonjet@mastodon.social makes a post, it will send a copy of the post to coolsite.example, which will distribute it to any of its users who follow @elonjet@mastodon.social
    • It’s important that only one copy is sent to coolsite.example, because if there were 100 followers on that server, you wouldn’t want to send 100 copies of the same message to the server. That could add up quickly, so instead coolsite.example keeps track of followers on its end and puts the message into all of their home feeds

There are other rules around boosts and locked accounts and all that but I’m trying to keep things simple so… maybe a post for another time.

The thing is, at this point there is one user on the server (me) following one user on another server (Elonjet). This is coolsite.example’s entire model of the Fediverse. So when I am curious about finding new people to follow and I say “show me every post with #kittens”, it shows me posts that use #kittens from me, and #kittens posts from Elonjet, and from other people’s posts that Elonjet has boosted with #kittens, and… that’s all. It’s likely that search results are empty even though there are probably lots of #kittens posts out there.

The “normal” way to grow a server’s model of the Fediverse and get more posts into it is for all the users on a given server to follow whoever it is they naturally follow, and then the server builds up a database of known accounts. Because of boosting, this isn’t even necessarily just accounts the user on a server follows. Over time, and with enough people doing enough following, a server eventually does learn about all sorts of corners of the network.

But if you are on a server of one person the model breaks down. It can be difficult for small servers to learn about the network.

A solution: relays

A relay is a piece of software designed to solve the above problem. A relay is basically a special kind of Fediverse server with a single “account” (though it’s a phantom inbox, not really an account, for complicated reasons). The way it works is it basically:

  • I decide I want more content to browse, so I google “Fediverse relays” and find a list somewhere

  • When I see a relay that looks active, I (as the admin of my server) subscribe to the relay via a special subscription page in Mastodon admin interface

  • Subscribing to the relay means we enter an agreement with the relay. What we agree to is: I will send the relay every public post made by my users, and in return it will send my server every public post made by every other server that subscribes to the relay.

:light_bulb: Note that this is about servers, not individual users. So if I hook my server of 1 user up to a relay and there is another subscribed server on there with 10,000 users, then that server now gets the posts from my 1 user, and my server gets the posts from 10,000 users. Most of those posts won’t end up in my home feed because I don’t follow any of them, but the posts will now live in a database so when I search for #kittens I will get any #kittens posts from those 10,000 users! Think of it as a commons of public content that everyone is going in on. You give content in order to get content.

Also important: there’s no guarantee that a relay will give you good content. Most public relays are hardly moderated at all, so it’s possible that you will hook up to a relay and suddenly see a bunch of terrible garbage in your feeds. As admin, you can always unfollow the relay on behalf of your server, and block/defederate it if it isn’t honoring the unfollow.

How to write software that subscribes to a relay

I have known about relays for many years but have never tried to write software that subscribes to a relay. I wanted to try it out and I thought there would be technical documentation somewhere but there wasn’t any I could find! I guess this makes sense, as relays aren’t standardized, they are just a tool that a bunch of people decided was a good idea and started making and using.

So I had to read a bunch of source code and do a lot of experiments to figure out the subscription step.

:light_bulb: Warning! There are two major relay subscription protocols (I told you it wasn’t standardized!). Broadly speaking there is a Mastodon-compatible protocol and a Pleroma-compatible protocol. Most relays support both but it’s important to know that there are two different methods. In this article I discuss the Mastodon-compatible method because that’s the flavor of ActivityPub I am most experienced with.

If you want to follow a relay, the first thing you need to know is the location of its inbox. This will be written on the website of a given relay and will be something like https://relay.example/inbox. (To further complicate things, some relays are private and will require you to email someone first to pre-approve you for this. But there are also public relays that accept anyone.) What you want your server to do is send a bare-bones ActivityPub Follow event to that inbox. That is, you will HTTP POST it like you’d POST a follow request to any normal ActivityPub inbox. The main difference is the content of the object field:

{ 
  '@context': 'https://www.w3.org/ns/activitystreams',
  id: 'some-long-unique-id-string-here',
  type: 'Follow',
  actor: 'https://coolsite.example/actor',
  object: 'https://www.w3.org/ns/activitystreams#Public'
}

To get really specific here:

  • @context is a string that says this is ActivityStreams (and therefore ActivityPub) data
  • id is a unique ID (unique to the server of origin — other servers likely use a combination of “domain + id" to disambiguate messages in case two servers happen use the same ID for messages)
  • type lets everyone know this message is an Activity Vocabulary Follow Activity
  • actor is a URL that resolves to an actor object that, importantly, has a property on it called endpoints which has a property on it called sharedInbox . The sharedInbox value is going to be the URL where you would like the relay to POST messages to your server. For a real-world example take a look at https://mastodon.social/actor, where the value of sharedInbox is (at the time of writing) "[https://mastodon.social/inbox](https://mastodon.social/inbox)" . The actor should also specify an inbox property. The sharedInbox and inbox can be the same, or different, it’s up to you how you want to handle messages.
    • For the purposes of this blog post let’s say my inbox is set to "https://coolsite.example/inbox" and my sharedInbox is "https://coolsite.example/inbox/shared"
  • object MUST be the string "https://www.w3.org/ns/activitystreams#Public"

So the actor and the object are the bits that are different from a normal follow request.

If you POST the object to the correct inbox, and the relay is a public one that doesn’t require you to contact an admin to get access to (or you have been pre-approved for a private relay), you should within a few seconds or minutes receive back to the inbox of the actor an Activity Vocabulary Accept Activity with the object you just posted to it embedded inside. In my made-up example, we can expect the following message to be POSTed to "https://coolsite.example/inbox":

{
  '@context': 'https://www.w3.org/ns/activitystreams',
  id: 'another-long-unique-id',
  actor: 'https://relay.example/actor',
  type: 'Accept',
	object: { 
	  '@context': 'https://www.w3.org/ns/activitystreams',
	  id: 'some-long-unique-id-string-here',
	  type: 'Follow',
	  actor: 'https://coolsite.example/actor',
	  object: 'https://www.w3.org/ns/activitystreams#Public'
	},
  to: [ 'https://coolsite.example/inbox' ]
}

This lets you know that the relay is going to start sending you stuff! You should log this somewhere so you can unsubscribe later if you need to (by sending an Undo/Follow). If you messed up somewhere, you might get a similar message but with a type of 'Reject'.

Pretty soon you should start seeing random ActivityPub messages POSTed to whatever you said your sharedInbox is (in my example, "https://coolsite.example/inbox/shared"). And that’s it: you’re connected and getting messages from accounts and servers on the Fediverse you didn’t even necessarily know existed. Do with those messages whatever you will: show them in a “federated feed” view, index them for local search if they have flagged that they want to be indexed, etc etc.

5 Likes