Proposal: Mechanism for synchronizing followers lists across servers

ActivityPub Server to Server
1

Context

Due to software bugs (e.g., Pleroma forgetting follow relationships a while ago, Mastodon mishandling Reject on Accepted Follows for the time being, etc.), crashes, backup rollbacks, etc., state may get out of sync between instances, in particular regarding followers information. With Mastodon’s follower-only privacy scope, this can get particularly damaging, as receiving remote instances are asked to deliver the message to who they know to follow the poster: if that gets out of sync, then the messages may reach people that the poster expects to not have access to.

Therefore, a mechanism to detect synchronization issues and correct them is needed. Such a mechanism has to take into account that the full list of an account’s followers may not be public information, and should thus only consider followers residing on a specific domain.

Proposal

Protocol extensions and an implementation are proposed in the following Mastodon Pull Request: https://github.com/tootsuite/mastodon/pull/14510

The basic idea being the addition of an optional collectionSynchronization attribute to activities such as Create and Announce activities. This attribute, typically set on activities addressed to the author’s followers specifically, would contain one or more SynchronizationItem objects with the following attributes:

  • object: the identifier of the collection this is describing. This collection MUST be owned by the activity’s author. For our current purposes, it must be the activity author’s followers collection.
  • domain: the domain name of the instance the message is addressed to. It can be used to catch implementation bugs or to allow implementations to inline multiple SynchronizationItem corresponding to different servers in a same payload (e.g. when using relays)
  • partialCollection: an URI to a collection that is the subset of the object collection made of the URIs of objects owned by domain. Accessing this collection SHOULD require the request to be signed (HTTP Sigs) with an account owned by the domain. For security reasons, it must be on the same domain as the collection it is describing.
  • digest: A Digest of the partialCollection's items, in alphabetical order, and joined by a newline. In the proposed implementation, SHA256 (http://www.w3.org/2001/04/xmlenc#sha256) is the only acceptable algorithm. Choosing a common algorithm enables caching that information on each instance as necessary.

A server receiving an activity with such a SynchronizationItem would check that it is the intended receiver (domain attribute), that the object is indeed the author’s follower collection, check that the partialCollection resides on the same domain name as the author, then check the digest, and in case that digest doesn’t match, it would fetch the partialCollection, remove any local follower not listed there, and send an Undo Follow for any follower listed there that it doesn’t know to be followers.

Example

Create activity:

{
  "@context": [
    "https://www.w3.org/ns/activitystreams",
    "https://w3id.org/security/v1",
    {
      "toot":"http://joinmastodon.org/ns#",
      "collectionSynchronization":"toot:collectionSynchronization",
      "SynchronizationItem":"toot:SynchronizationItem",
      "partialCollection": "toot:partialCollection"
  }],
  "id": "https://social.sitedethib.com/users/Thib/statuses/1234",
  "type": "Note",
  "attributedTo": "https://social.sitedethib.com/users/Thib",
  "to": ["https://social.sitedethib.com/users/Thib/followers"],
  "content": "This is just a private toot",
  "collectionSynchronization": [
    {
      "type": "SynchronizationItem",
      "domain": "mastodon.social",
      "object": "https://social.sitedethib.com/users/Thib/followers",
      "partialCollection": "https://social.sitedethib.com/users/Thib/followers_synchronization",
      "digest": {
        "type": "Digest",
        "digestAlgorithm": "http://www.w3.org/2001/04/xmlenc#sha256",
        "digestValue": "b08ab6951c7d6cc2b91e17ebd9557da7fae02489728e9332fcb3a97748244d50"
      }
    }
  ]
}

Result of querying https://social.sitedethib.com/users/Thib/followers_synchronization (when signing the request on behalf of a mastodon.social account):

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "id": "https://social.sitedethib.com/users/Thib/followers?domain=mastodon.social",
  "type": "OrderedCollection",
  "orderedItems": [
    "https://mastodon.social/users/Gargron"
  ]
}