Hi, I’m currently working on groups support for Mastodon.
The definition of what a group exactly is unfortunately differs from project to project, but what we have settled on is:
groups have a set of members and administrators/moderators (we’re assuming this is public information, but that may not be a hard requirement)
only group members can post in a group (this one isn’t set in stone)
posts and replies within a group all have the same audience, set by the group (group members, and potentially publicly-viewable as well): no per-post scopes like we currently have in Mastodon
similarly, mentioning non-members, if at all possible, should not extend the audience of a group post
a group post is bound to a specific group and cannot be cross-posted
posts are viewed in a per-group timeline and not on the home timeline
new members may or may not need to be approved (group-specific settings)
moderators can kick/ban group members and delete individual posts
ideally, users should be able to report group posts not only to server moderators but to group moderators as well, and have a choice in that (e.g. be able to chose not to report to group moderators in case they are wanting to report the group’s behavior as a whole to their server’s moderators)
For implementation complexity, UX and moderation reasons, I have decided on the following restrictions, at least for the initial implementation in Mastodon:
no Announce functionality whatsoever within or across groups
mentions of non-members won’t be processed at all. This is mostly an implementation detail, though it may have UX implications
only groups with publicly-viewable posts will be supported. While this may change in the long run, restricting ourselves to publicly-viewable posts for now avoids some potential technical difficulties and moderation issues.
I have not started writing the ActivityPub side of things, but what I have in mind is roughly:
group actors have Group type
Join is used to join a group, and Leave to leave it. It follows the same flow than Follow, with Accept or Reject being sent either automatically or after manual review
both top-level group posts and replies are sent only to the group, which is responsible for distributing them (they may be publicly dereferenceable, but distribution itself is handled by the group actor)
posting within a group (at least for top-level posts) uses FEP-400e or similar
This seems roughly consistent with what has been described here for Smithereen, lotide, and @macgirvin’s projects, so I’m hopeful we’ll manage to have something interoperable.
That being said, there are still quite a few open questions, even with the limitation I listed for Mastodon’s initial implementation:
How do we signal the audience of group posts?
If we use to/cc/audience for this, how do we ensure non-group-aware implementations do not interact with them?
If non-group-aware implementations do interact with them, what do we actually do? Drop the interactions? Handle out-of-group replies to group posts? This is something we would very much like to avoid in Mastodon for UX reasons.
Posting by sending to the group only, with the group deciding whether to allow the posts or not, seems reasonable. But what about other activities, like editing a post or deleting one? A group actor preventing someone from deleting their posts does not sound good, especially since the person deleting their post will have no way to know if the information is distributed to group members.
how should posts from someone who has left/been kicked out of a group be handled? should the author still be able to access them? how? what about replies?
How is group membership kept in sync? Are Join/Leave/Accept forwarded? Signed? Publicly dereferenceable? Is a collection regularly polled?
I also have other issues in mind, although they don’t apply to the reduced scope of Mastodon’s initial implementation:
how do we distribute private group posts? One proposal is Smithereen’s “actor tokens” (short-lived tokens issued by the group actor to its members), although I’m not sure how well it would scale
how do we handle federated moderation (having moderators from different servers)? This seems more like a vocabulary issue than a design one, but I haven’t given a lot of thought yet.
IIRC the Mastodon plan is to not have mentionable groups at all. Instead, Mastodon groups exist in their own context and all posts/replies are hard-coded addressed to the group. They are completely separately scoped, and will not show up in the main timelines at all.
As far as Webfinger is concerned, @groupname@domainmay be required, but there is less reason to do this compared to user profiles.
This is an implementation detail. For Mastodon, it probably should be.
*yet? I imagine this should be possible in the future, especially if Add is used instead of Create.
re: initial implementation, this also makes sense, bearing in mind that it is initial implementation.
Group type is fine, but perhaps some extra extension properties or types should be used to signal how the Group works on the backend. There are some existing projects that already use Group actors for mailing-list-like use cases. Some thought will be necessary to differentiate those “groups” from Mastodon-compatible “groups”. The use of @type = as:Group is not enough.
There needs to be a way to signal that a Group is a “joinable” Group and not a “forwarding” Group. Or it may be both. Perhaps an extension type like toot:Group instead of as:Group? Perhaps an extension property like toot:joinable?
I somewhat favor the first of these two options, as it makes clear the distinction between the typical ActivityPub delivery model (Follow, inbox forwarding, get from outbox) and the proposed mechanism here (Join, distribute an Add, get from wall). Projects that support both models may declare a type array like @type = [as:Group, toot:Group]. This would signal that the Group may be both Followed and Joined.
The downside to using a property like toot:joinable instead is that any implementation that does not understand this property may attempt to send a Follow rather than a Join, which may not be understood. But on the other hand, it is more generic, and could be used to signal (for example) that an as:Organization follows the same Join-based semantics.
It may actually be a good idea to use both.
Join, Accept/Reject Join, Leave, that’s all fine too.
Sending posts and replies to the group for redistribution is fine too and probably necessary for the proper UX. But there should be some way to signal that a post is authoritatively owned by the Group, maybe? FEP-400e (or similar) is certainly one way to do this, as a group “wall” is assumed to be managed by the Group actor, and Add/Remove into that “wall” simplifies both distribution of activities and also avoids semantic issues about what it means to Delete someone else’s object. My only concern with FEP-400e as currently written/finalized is in its use of “abbreviated objects”, which doesn’t seem JSON-LD compliant to me.
re: open questions:
as:audience can contain either the Group IRI or an array of Group and as:Public. There is a slight pitfall in that some implementations may copy audience into to/cc and therefore copy as:Public, causing a group post to “leak” into the regular timeline for Mastodon 3.5 and below, but this is why I favor using an extension type as described above. Non-group-aware implementations will probably ignore the existence of toot:Group entirely, no?
Implementation detail: drop all interactions from non-members (inside the group context). If a post leaks out of the group context, then those interactions may be delivered to the inbox of a regular Person actor, but this is not much different than the issue presented by blocked actors/domains, or by non-reply-control-aware implementations in a FEP-5624 world. There is always the chance that a separate conversation may be taking place outside your control, and all you can do is ignore it or filter it out.
Sending a GET to the group wall with an HTTP Signature should return at minimum any posts authored by the owner of the signature. If the signer is also a member, the GET should return all group posts as well. This should allow prior members to view their old posts and delete them if they wish.
Alternatively, you could say it is the responsibility of the individual actor to keep track of which of their posts are in which groups.
Replies should be visible if they are publicly dereferenceable via the replies collection?
Add/Remove to the members collection, if it matters (and group membership is publicly visible). The members collection should maybe use the same HTTP-Sig mechanism as the wall, so that anyone can check if they are a member by whether they are included in the partial Collection returned. Or you could rely entirely on keeping track of the history of Join / Accept Join. There’s also your follower collection synchronization FEP that could help in shared server scenarios.
Actor tokens could work, bearcaps could work if fetching the wall is the primary method of interfacing with a group. And one other option is, if you implicitly trust the group to verify everything and not commit forgeries, then it can forward activities without a signature. But for verification it would probably be a good idea to use bearcaps. I’m not sure what that implies for scaling.
Either with Offer Remove, or as an implementation detail you could have the Group send out a new Remove attributed to itself if it was received from a member with the appropriate permissions.
Looks best option of the two, and has positive side-effect that it’ll encourage fedi apps to support type arrays. However, wondering about toot:Group. The namespace prefix “toot” more or less indicates: This only applies to a Microblogging domain. But what you are modeling is Membership. A generic concept.
This specification intentionally defines Actors in only the most generalized way, stopping short of defining semantically specific properties for each. […] External vocabularies can be used to express additional detail not covered by the Activity Vocabulary. VCard [ vcard-rdf] SHOULD be used to provide additional metadata for Person, Group, and Organization instances.
This ontology is designed to enable publication of information on organizations and organizational structures including governmental organizations. It is intended to provide a generic, reusable core ontology that can be extended or specialized for use in particular situations.
This ontology is widely used, well understood. Though it has kinda of an intricate model… it defines Membership, Roles and more:
Now I don’t know how that would look like as AS msgs. Where the diagram says FOAF we have Actors. We might have @type = [as:Group, org:Organization] and some membership construct to indicate a “Group with Membership” (group and organization aren’t disjoint). In a similar way there may be a use of org:Role to model - in the case of a Microblogging domain - Moderators and Admin members. (While e.g. in a federated forge app the same ontology is used to define a role of Project Maintainer).
Biggest downside to me seems that now you are using a custom property to basically state that this is a different type of Group… one that supports Membership. Within the context of the Microblogging domain of “toot” vocabulary that may be acceptable, if everyone consents with it, but to my untrained eye it looks not to be a mechanism that scales well for broad interoperability (though a combination of a toot:Group in the type array with this property may be perfectly fine).
no, the toot namespace has no such meaning. it means nothing at all outside of being a shorthand for the https://joinmastodon.org/ns# namespace, which is for vocabulary defined by the Mastodon project.
in any case, the namespace really doesn’t matter. we just can’t use the as namespace because there’s no proper viable path for extending the ActivityPub core spec at this point in time. toot is fine for Mastodon’s purposes.
If there was some kind of organization or working group that could adopt extensions into a common namespace, then sure, you might have a point, but right now the development of extensions is led by individual projects and implementations. There was LitePub for a while, but that was more about dropping JSON-LD and adopting HTTP Signatures. Perhaps the FEP process could manage its own JSON-LD context file, which could make it easy for projects to adopt all the FEPs at once by simply including one additional context. But this isn’t strictly necessary. It would also save having to add toot, pixelfed, sm, misskey, and so on, though… and it could be a more neutral/agnostic way to handle extensions that are developed by multiple projects in collaboration, so that no one project claims the definition unilaterally. But this isn’t really important or significant at the end of the day.
(Apologies, I have injured my hands yesterday and I might write less and slower than I might have wanted)
Group posts are not going to work by mentioning a group but by selecting the group in the UI, so there is no specific syntax for mentioning a group. However, at least in the current state of my (unfinished and unmerged) implementation, groups are actors that are in the same namespace as users, and also require webfinger (we might be able to lift this restriction, but it would require changes i haven’t started working on yet), so searching them would use @group@server.
I am not sure. I think it’s way easier to reason about both UX and protocol if posts are restricted to only one group. It also falls in line with your own concerns:
I definitely agree here, and your suggestion of toot:Group seems fine to me, although I’m not sure whether that should be a toot thing (especially since, annoyingly, the toot namespace points to the general project website, not any protocol-specific page with readily-available documentation).
How would the toot:Group not being recognized prevent the posts from being processed, though? Since the posts themselves would be regular as:Note or similar objects, and as:Public would be in the audience, the recipient does not need to know how to handle toot:Group to process the post.
Yes, that’s why, as much as possible, I want group posts to be incompatible with group-unaware software.
Yeah, inbox forwarding seem like the natural way to do it, but this means you rely on the group actor distributing it, which you can’t guarantee. A malicious group actor could also send you an Accept and not forward anything. Forwarding also requires either LDSigning or making the whole activity publicly dereferenceable.
This absolutely doesn’t have to be done on first iteration, but I imagine crossposting in groups to work much like Reddit crossposting. You could arguably do this with an Announce and then reply to the Announce instead of to the original Note, kind of like how in really early Mastodon you could reply to boosts due to a bug. I’d much rather not worry about that right now, though. But I will point out that FEP-400e’s answer to this is to use the target property on the Note itself, to signal that the Note is part of the wall collection? I still have concerns about 400e’s use of stub objects and how that interacts with JSON-LD processing, though. So there might be a better solution here. I kind of wish we could use partOf from the CollectionPage properties here… I’ll think about it later.
There was some later discussion in this thread about possibly putting things under an FEP-managed namespace that would be defined by a proper JSON-LD context document, perhaps that would be worth pursuing at some point? That context document could also alias existing properties for compatibility…
attributedTo as an array could include the Group? IDK if this is the best solution, though. And maybe it’s not the kind of thing we can entirely prevent, as it is always possible to author an activity and deliver it to both a Group and to other recipients.
I mean, you can’t guarantee anything as soon as there are other actors and other software involved. But if “the group is responsible for distributing everything” is an already-decided-upon constraint, then you don’t really have much choice.
Depends on how much you trust the Group actor and the software it’s running on? It may be that someone’s security model allows for implicitly trusting the Group and its software to not be malicious, although having better verifiability is always nice, and you can get that verifiability by fetching with bearcaps I guess as a 3rd option.
re: “What about replies to posts by people who have left a group”: If the replies collection is provided and publicly accessible to non-group members, then replies are visible. But the person who left the group will not be notified of new replies, because the Group will not deliver Add activities or forward anything to them. Unless some software addresses the author of the post in addition to the group, that is? The challenge seems to be in deciding whether or not the ex-member’s replies should be distributed to the group members or not. If not, then it would be possible for people to reply to an old post, but the author of that post wouldn’t be able to respond in anything other than what is effectively a regular DM outside the group. If yes, then that’s an exception to “only group members can post in the group”.
Yes, that’s what FEP-400e suggests (SHOULD), and that seems fine to me.
It could work, though the semantics sound a bit muddy to me: would that mean the group is a co-author of the post? Is that something we want to represent this way?
Well, we’re discussing making group-unaware software not process the object so that even if they receive it, they wouldn’t interact with it. Now, if someone deliberately crafts two versions of the activity or something, I don’t think it’s a situation worth concerning ourselves with.
re: 400e and target on objects: nothing new to say, haven’t thought of anything better yet, it might still be enough to have addressing to the group and forego the use of target unless it provides something useful.
re: delivering to a group as well as additional recipients: i meant more that you could send one activity that would be interpreted differently in aware and non-aware impls, and the “fallback” would be a regular status. that’s not technically impossible or disallowed, to post to a group and to your followers (in a non-Mastodon implementation, anyway). it might even be explicitly intended to have the Note play “double duty” in that way. you could also author an activity representing posting into multiple groups. i don’t think there’s a way to restrict such usages – we can only recommend against them with SHOULD / SHOULD NOT, if it’s deemed worthwhile.
yes, “every proposal so far forbids that”, but there is no mechanism by which this can be enforced. addressing is up to the sender.
re: attributedTo arrays including the Group: yes, it would imply semantically that the Group co-authored the post. this may or may not be a good idea, given that it might allow for Create/Delete/Update/etc based on the authority of the Group rather than the original author.
re: bearcap verification flow, it would be similar to the “public dereferenceable” option, but with fetching a bearcap URI instead of an HTTPS URI. It’s essentially just one step below public, since you need a token (which is provided by the bearcap, much like leaving a key under the front doormat). This could be “good enough” for non-pirvate groups. I imagine private groups will want LD Signatures or something similar, as they would want to not be fetchable under most (if not all) circumstances.
I mean we’re defining extra vocabulary, its meaning and their expected behavior. Of course we can’t prevent an implementation from producing documents that go against what we define, but that seems beside the point? We’re defining group posts, and I’m talking about defining them in such a way that implementations that do not support this extension do not fall back in an unexpected way. We can decide to require group posts to not “play ‘double duty’”. Of course an implementation could decide to do otherwise but it would then not be compliant with our extension and should not expect its behavior to be supported.
I’m not sure what use bearcap would be for public posts. I’m also not sure how it would help you with verifying that a Delete has been properly distributed to group members. What would you even try to fetch to make sure of that?
What form of language would this take? “Implementations MUST NOT address actors other than the group”? Or would that be a SHOULD NOT? “Implementations SHOULD NOT address actors other than the group, as this would cause undesired behaviour in non-aware implementations”? If it’s a SHOULD NOT, then that’s basically what I was suggesting earlier – a recommendation for behaviour. I can still see how someone would want to post into a group and also to their followers, though they would have to be aware of the consequences of such an activity.
In actuality, I think it might not be so important to forbid this or guard against it, if the undefined behaviour can instead be defined. You could say that the group has its own rules for distributing posts and replies (to members only), and this can coexist with the classic direct-distribution method. There will just be some replies that the group will not be aware of, and there will be some replies your followers are not aware of, and that’s probably okay.
The relevant required behaviour for the group is to distribute anything received from its members, to other members. As long as it does that, is there necessarily a problem with anything else? After thinking about it a bit more, I’m inclined to say that there isn’t. The only issue is that if Mastodon wants to ensure its expectations are met, then other implementations SHOULD NOT address to other than the group. And this issue is basically only an issue because of prior versions of Mastodon checking for as:Public as the first thing when attempting to determine scope/visibility.
Maybe it would be a good idea for group-aware impls to adopt the zot/nomad replyTo property? This way, if they see the replyTo property present on the object, they should completely ignore to/cc/audience/inReplyTo when calculating the addressing, and just copy the value of replyTo verbatim into the to property of the activity. Essentially, this would allow group-aware impls to not leak posts to non-aware impls.
The potential downside to replyTo is that you would need to add some logical checks so that it doesn’t get abused. I can imagine someone crafting an activity with a replyTo set to someone else completely unrelated, and implementations start DM-spamming that other person. I’m not sure which logical checks make sense here…
Maybe the value of replyTo must be either in attributedTo or audience? That would still allow setting an audience of someone else and replyTo of someone else, though. Definitely needs more thought here.
Maybe this should be combined with FEP-400e and the logical check would be if replyTo == target.attributedTo? That would leave the usage of replyTo on its own as undefined behaviour, though… Maybe while defining replyTo we could just have a generic “implementations MAY apply their own spam-filtering policies”?
Maybe instead of a logical check, this should be exposed in the UI to the user, so they know who they are replying to? Then, the user can decide if their reply would be “spammy” or not. (I think this is how Reply-To headers work in email.)
Maybe the property isn’t necessary, and all you need is the SHOULD/SHOULD NOT guidance on addressing the group?
Is distributing the Delete necessary, or is it enough to Remove the post from the wall? I think we still need to figure out how posts should be distributed by the group. I was assuming that the primary mechanism would be Add/Remove to wall, and I’m not entirely sure how to handle deletes and edits.
There’s two parts here: the first is what to send to the Group, and the second is how the Group should respond.
If you send a Create to the Group, the Group should distribute an Add to its members.
If you send a Delete to the Group, the Group should distribute a Remove to its members.
If you send an Update to the Group… I’m not sure how the Group should respond here. Maybe Announce Update?
Actually, there’s another thought I just had:
If the Group is the one sending the Adds and Removes, does FEP-400e even matter anymore? The wall collection doesn’t need to be “publicly appendable” if the owner is the only one appending to it.
And another thought:
Is there any value or use in Following the Group or checking its outbox? How much sense does it make to both Join and Follow a Group? I imagine these would be separate controls, so that non-members can follow public groups to see what’s going on in the group, but without being able to post into the group. But if that was the case, then why even have them be separate? Why not have membership in a group be Follow-based like with current mailing-list-style Groups? I think the only reason we’re using Join right now is to explicitly break compatibility for non-aware implementations, right? But don’t we already get that break already, simply by using Add/Remove for distribution, with non-aware impls not knowing what a “wall” is?
I think having type = toot:Group and having your outbox be full of “Add/Remove to wall” activities might be enough to guarantee a separated UX. This way, non-aware impls will fail to parse the actor when resolving them, and even if they did manage to resolve and successfully Follow a toot:Group, they wouldn’t know what to do with any of the activities they received unless they added support for walls. There’s no need to break from the “mailing list” metaphor if it still works. It would just be a slightly different interpretation of it (with Add/Remove instead of Announce).
That would lead to a fractured and confusing user experience, especially in a “posts and comments” approach. Another issue is software that does not understand groups but will do so in a subsequent version: for instance, if current Mastodon would be to understand “group posts” as regular public posts, there would be nothing in the database that would allow a migration to re-interpret it as a group post. As a result, once updated to group-aware Mastodon, the instance would behave differently for group posts and their replies based on whether they were discovered before group support was added, or after the fact. Some other software does store enough information to possibly re-interpret the posts after an update, but I’d wager that even in these cases, the sheer logic and computational complexity required for doing that would be prohibitive.
The same question applies if it’s a Remove. How do you ensure the Remove has been distributed to other group members?
Join and Leave seem more semantically explicit than Follow and Undo Follow, I’d say the only reason to use the latter is to keep compatibility, but we want to explicitly avoid that, so…
Considering the requirements for Mastodon’s initial implementation to:
have groups and group posts be incompatible with current Mastodon versions
have the group forward member posts around
ensure all groups posts are expected to be public
What about the following proposal?
MUST have type: "toot:Group" (the intent is to explicitly break compatibility with existing Mastodon versions)
MUST have a sm:wall collection
MUST have an audience attribute with either as:Public or the members collection
MUST support Join and Leave (replying with Accept or Reject)
MUST have a sm:members collection that SHOULD be dereferenceable by at least group members
SHOULD have their administrators listed in attributedTo
There should also be something to tell whether members are to be manually approved, and something to tell whether non-members are allowed to post, but Mastodon can still make safe assumptions which will only degrade UX a little if they don’t match reality.
Group posts (and replies)
MUST be sent using FEP-400e using the group’s wall as the target
MUST have an audience set to the group’s members collection (for group-private posts) or an array with (as:Public and the group’s members collection). The purpose is to make sure the post is intended to be public
MUST be attributedTo the group actor and the post’s author (the intent is to explicitly break compatibility with current Mastodon versions)
To be honest, I am not too sure about using attributedTo for that, but it seems to be a good way to break compatibility with the current implementation. I’m also not sure about using sm:wall for replies: it sounds like it’s not what posts-and-replies implementations would want, but I’m not sure any other mechanism has been proposed for that.
One interesting issue is what should happen if a group’s privacy model changes other time? If such a thing is allowed, and a group switches from public to private, what does it mean for existing posts? And what does it mean for the Mastodon’s initial implementation plan of only handling public groups?
if the group goes “private” then it could be disabled in mastodon? “This group is a private group. Mastodon currently only supports public groups.” And then grey out the Join button, and disallow current members from posting into the group? Optionally, do not show the wall. If/when Mastodon supports private groups, simply re-enable the UI around them.
On a protocol level the only practical consequence is whether the sm:wall[items] is publicly accessible or not (and similar logic extends to sm:members too, probably).