By means of Flarum maintainer Daniël Klabbers an announcement was made that v2.0 of the Flarum forum software will have federation support. The federation support is made possible with help of a NLnet grant.
More information is on their main forum:
1 Like
FYI, I also informed #software:discourse forum, where @angus is posting about the work that The Pavilion is doing to add federation support via Discourse plugins…
Ideally the 3 forum implementations for the Fediverse should coordinate and submit #standards:fep documents where needed.
4 Likes
it would be nice to be able to browse different threads as contexts (7888 tries to describe this, but most projects currently rely only on inReplyTo and have no real way to group related objects and activities)
i am not entirely sure if the named projects here are going to build out an actual data model for forums though… sadly, it seems like they are all interested in mastodon compatibility first and foremost, which means trying to cram everything into a distributed profile-shaped hole instead of having a browsable thread-based solution.
i suppose i might as well put this out there: any forum interested in modeling a proper forum is welcome to reach out to me for consultancy on how it might be done. i’m kind of already doing it for myself anyway, just haven’t gotten around to fully documenting all the necessary pieces (which includes things like contexts and collections and adding/removing into them rather than creating/deleting only, but there’s more to it of course…)
4 Likes
Hey guys, author of the (soon to be released) Discourse ActivityPub plugin here. Firstly, great to see some other views on this!
Based on my (admittedly short) experience with ActivityPub so far, I’m not so sure. I know you know this, but it’s worth pointing out that ActivityPub is (mostly) a communications/networking protocol, which means that it’s not determinative of the data model used by its clients. Indeed, over-engineering a communications/networking protocol has its own risks.
The “forum” / “social media” distinction breaks down a bit when you really get into it with software like Discourse, something I found when making the “follow” plugin for that framework (it was acquired by Discourse), and which has been reinforced for me while working on the AP plugin. Posts and their replies can exist outside of the context of “topics” (aka threads) relatively easily.
I can definitely see the utility of having an additional property in the AP vocab for topics/threads, as an extension (maybe there is such a property in some proposed extension already?). But I’m not sure the AP data model itself would need to be fundamentally different.
2 Likes
the data model is context as far as i’m concerned. the difference between a forum and social media is just in whether it recognizes context or if it instead is fine with context collapse. you might consider the difference between a Create and an Add. the former does almost nothing, the latter explicitly manages a collection representing the “thread” or “topic”. one might even consider a profile to be an explicit collection, although mastodon et al do not.
EDIT: linking to a thread where this was touched upon with regards to nodeBB. infinite love ⴳ: "@devnull@crag.social i'd probably interpret a con…" - Mastodon
1 Like
Could you explain your thinking on why a topic (let’s use that over thread) needs to be in a collection?
For context on my thinking on this the first version of the Discourse AP plugin creates Group actors for Discourse categories, similar to what Lemmy does. Within that setup you can create topic structures using inReplyTo.
You could also have various variations of actors within a forum system, e.g users, tags or topics themselves. The AP data model is not necessarily determinative of the client data model.
I guess you could use context and define a more granular AP forum communication standard, but I guess I’m just wondering why? What advantages do you see in that? What problems is it solving? Apologies if I’m just being slow on the uptake!
1 Like
it doesn’t need to be in a Collection, but conceptually you are probably going to be treating it as a pseudo-collection if it doesn’t exist. the lack of a “real” object representation means you miss out on a lot of things that would otherwise be useful or perhaps even crucial. consider:
- how do you represent a topic title without an actual explicit topic?
- how do you represent inclusion in a topic without an actual explicit topic?
- how do you represent moderation on a topic without an actual explicit topic?
- how do you represent two different topics that are part of the same reply tree?
- how do you represent topics where none of the objects are in reply to each other?
and so on. all of these questions might have workarounds, sure, but they’re going to be messy because they’re all implicit and therefore ambiguous. you might defer to some other indirect property, like the name or attributedTo of the object that is at the root of a reply tree (which you might not have, btw). but this is granting the “root post author” undue authority that they don’t necessarily have. you might honor a Delete against a child reply in your conception of a “topic” formed implicitly by reply-chains… but someone else might not. on the other hand, a Remove from an explicit Collection that is explicitly attributedTo you? that’s unambiguous.
it is because the client data model is so variable that we must therefore try to keep the AP data model as generic as possible, where the semantics of each property are kept clear and simple. inReplyTo means specifically a reply: “i am responding to this”. context should mean specifically a context: “here’s the logical grouping of related stuff”. (this is the “intended usage” which for reasons beyond me were left “intentionally vague”; FEP-7888 tries to demystify that and make it less vague.)
consider if something like Discord were to use AP. inside each topic/channel/etc you have a bunch of messages, and “replies” are just metadata, rendered as a small one-line preview above your message. would you force every message in a channel to be inReplyTo some other message? that’s unnecessarily fragile and overly strict. the only thing binding those messages together is the “context” of being in the same channel. the chain of replies is a completely separate and parallel structure to the channel context.
and really, there’s no difference between a “channel” or a “topic” or a “conversation” if you follow this line of thinking. they’re all just objects in contexts, some of which might be in reply to others. it’s the presentation that makes all the difference. you have generic data with specific presentations as a “forum”, “chat app”, “social media”, whatever.
the following list is non-exhaustive:
- forums as collections of topics
- forums that can include subforums
- forums with actual metadata
- topics with actual metadata
- topics with moderators separate from the author of the first post
- tracking a topic separately from replies
- not having to reconstruct a potentially unbounded reply tree via recursion
- not panicking when you encounter a deleted post in the middle of the reply tree
- having a topic be in reply to another topic
- having a topic be in reply to a post from another topic
- having a topic be in reply to a blog article
- starting a new sub-topic in the middle of another topic (e.g. “threads” on discord)
- private topics that you can add and remove participants
- private forums and subforums that you can add and remove participants
…and many, many more.
1 Like
You get the topic title by walking up using reply_to until you hit something like a Page (or a few other object types in Lemmy which indicate a topic). Its really simple and unambiguous. In fact adding another way to specify the topic would make it ambiguous. You cant compare this to Discord. There is no generic data and its not fragile, but strictly typed. Anyway this discussion is moot as long as no project in the real world federates like you say.
1 Like
- the Page type and other objects types you mention only “indicate a topic” in Lemmy and other projects that behave like Lemmy. outside of that, they indicate an actual web page, or whatever their defined semantics are.
- “walking up using
inReplyTo” is going to break when you hit a deleted or private object. - you can indeed compare this to Discord. there’s no difference between “messages in a channel” and “posts in a topic” because they’re both objects in a context.
- strict type-checking and strict data models are inherently “fragile” because they can be broken more easily, and they will be broken. it’s a question of when, not if. extension activities, type arrays, actors that aren’t one of The Five Types… all of these things are going to happen, and they’re going to break current implementations hard. the less we assume, the better.
basically, leave the domain-specific assumptions within the presentation layer, don’t let them muddy up the data layer. broadly speaking you should be able to deal with generic objects and generic collections as described in the AS2-Core data model.
or, in other words:
no one should have to care what your client data model is. the AP data model (really AS2) is generic enough to allow building anything on top of it as long as it adheres to the “activity stream” or “stream of activities” mental model. i’ve somewhat jokingly said before that “real activitypub has never been tried”, and it’s because of things like this. people are building their own protocols on tops of bits and pieces of activitypub, with very specific assumptions and requirements baked into the data model, and the protocols only overlap sometimes. the data model is effectively set by the client protocol and anything you send over activitypub will be transformed and coerced into the client data model instead of existing as a generic data model for the entire social web. activities start to look and function more like an RPC request than an RDF resource.
1 Like
This is great to hear!
@trwnh I appreciate where you’re coming from.
I spent a fair chunk of 2022 on a project focused on creating a more workable data model for sharing event data as it relates to the (outdated) iCalendar ( i.e. RFC 5545) and emerging JSCalendar/JMAP standards. I’ve also done some work in the verifiable credentials space which, through the medium of the W3C Community Credentials Working group has ongoing debates somewhat similar to what we could have here. Indeed, just between friends, I would sometimes read the knock-down drag-out debates about how best to improve different aspects of the VC standards on their group mailing list as some light daily relief 
The dynamics at play in those contexts, and in this one, is a combination of the following:
-
When you’re building something in the context of a standard that involves a variety of clients and implementations you have to start with the existing standard, as flawed as it may be. There’s no way around this. Yes there may be some interesting ideas about how that standard could be improved (there are many in the calendaring and credentials worlds too), and perhaps improved for your specific context, however it’s simply a fact that those ideas are not the standard and the standard is the standard. As tautological as that is it gets at the fact that part of what a “standard” represents is a process of compromise and debate over months and years, responsive to ongoing developing usage.
-
Changes to the standard are going to happen slowly, through much debate and “real world” usage. No matter how attractive an idea seems on paper, there will be debates and there will be many unworkable parts of it that need to be hewn before it becomes, well, a standard. It’s no coincidence that both Discourse and Flarum (and others) are joining the fediverse not long after the changes at Twitter. The choice to initially ensure Mastodon support is not hard to understand in that context.
What you’re suggesting could eventuate in some form, but it’s not going to happen overnight and it’s probably not going to happen in any fulsome way in any of the new forum implementations mentioned in this topic, i.e. the Discourse and Flarum implementations (I say probably because I can’t presume to speak for the Flarum implementation).
You might be right, you might be wrong, but the overall process and how it’s responsive to practical usage is what really matters here. Don’t get me wrong, I think you have some interesting ideas and I don’t want to pour too much cold water over them. The extent to which we have a real discussion on them is going to be relative to the process through which that occurs, the time demands of the participants and how the real-world implementations of the standard in the forum context are working (or not).
I don’t want this to come across as “it is what it is” because I don’t want to be cynical, defeatist, or condescending for that matter. My point is more that to do something like you’re suggesting there would need to be a robust process with the right participants and for each point to be discussed in much more depth. Then maybe in like 2-3 years something might eventuate. Or it might not.
1 Like
the tl;dr of it is, “if you want to support mastodon, just stick to Create Note and Announce Note”. if you want to support lemmy, make your actor a Group and make your first item a Page. that’s besides the point, though.
the real point is that you could get most of the improvements right now by using a single property: context. if it’s present, use it for logical grouping as intended. if not, do whatever you would be otherwise doing with the reply tree. and hey, if it resolves, maybe make it resolve to a collection representing the topic? (Streams) does this, more or less.
it’s just that, once you make this single leap, you start to realize the advantages of having an explicit collection rather than an implicit one. and this is something that in my opinion is crucial for anything that wants to call itself a “forum”. what is a forum without actual topics?
so it’s less “the standard could be improved” and more “the standard is broken”. there are very real problems that all stem from not having a real context. my non-exhaustive list above has over a dozen points. the most prominent problems are the ones that have to do with crawling the reply tree exclusively – notably, when you encounter a deleted or private or unresolvable object, there’s pretty much nothing you can do. projects like smithereen will throw away the entire subtree if they can’t find a “root post”. that’s a pretty big failure mode! and it’s not the only one.
to quote myself from FEP-7888’s intro section:
the general pattern that emerges is that a context exists to form a purposeful grouping, regardless of the specific purpose. For example, if we had the notion of a conversation, then we might reasonably say that someone owns this conversation and can apply their authority to it. Looking at some object or activity within this context is generally not recommended on its own; it is better to view the entire conversation or some page of it rather than viewing a singular message.
what you lose by ignoring context is, well… context. you are assuming authority outside of any explicitly defined scope. some projects will agree with your assumption that the root post has authority over all child replies. some projects will disagree heavily with that assumption. this leads to a divide between “facebook-like” and “twitter-like” models. but why assume? why not make it explicit? it actually ends up saving you the work of having to figure out how to group stuff later!
1 Like
I appreciate you feel strongly about this and I can see some benefit in what you’re proposing. I have read fep-7888. We’d need to discuss a number of the points you make in some detail before adopting it as an approach. Yes, it’s just one property, but, as you’ve also alluded here, you also have a larger architecture you want to apply to specific contexts through that property.
Implementers of a standard can’t assume the standard is broken. In my (limited) experience working with implementing the ActivityPub standard the past few months, I wouldn’t agree it’s broken. Yes, I have a lot less experience with it than you do, but you can’t start from that assumption, particularly when existing implementations such as Mastodon and Lemmy adopt the standard in its current form.
If the existing ActivityPub community wants to influence the development of the practical implementation of the standard in the forum space the way to do that is through improving the process of how the different implementers talk to each other, i.e. to play a facilitating role.
1 Like
perhaps instead of “broken”, would you accept the wording “has significant shortcomings”? because most of the “breakage” revolves around replies and deletes, and a lot of work is wasted reconstructing something implicitly with less expressiveness and functionality. you end up pretending there’s a collection later anyway, but you have none of the ways to actually manage a collection.
problem a: moderation
say you want to get rid of someone else’s post in a thread. do you send a Remove or a Delete?
currently, lemmy will send a Delete wrapped in an Announce, and mastodon will reject it for two reasons: one, because it’s not a valid reshare (the meaning of Announce is to “boost”, not to inbox forward), and two, because the lemmy community’s actor doesn’t have the authority to delete other actors’ resources (they don’t own the object, it’s not attributedTo them and there’s no clear hint that they would have this authority).
if, instead, there was a collection that had its own attributedTo and could be explicitly targeted by Add and Remove, then this ambiguity is entirely eliminated. you might not have the authority to create or delete someone else’s object, but you do have the authority to manage your own collection.
problem b: broken reply chains
how do you group together posts in a topic?
well, if you say that the reply tree is the topic, then you need to do a lot of work to back this assumption up. upon discovering a post, you need to follow the reply chain up until you figure out which pseudo-topic it’s in. and again, if you encounter a deleted or private or unresolvable post: what do you do? you don’t have a way to discover the “root post” that is holding this all together and is the basis for all your implicit assumptions. do you throw away the entire subtree as wasted work? do you create a new topic for that subtree?
meanwhile, in context-aware land: if there’s a context you group it with that context. the topic it’s in is the topic that’s explicitly specified. no assumptions necessary.
problem c: “that’s not what reply means”
again: if you treat the reply tree as the logical grouping, you are closing the door to actually replying to something. the “reply” cannot do double duty as both grouping and responding. if you have a flat structure, you end up having to force it into a tree shape.
the alternative proposal is much simpler: use context for logical grouping and use inReplyTo for replying.
problems d-infinity how we can progressively enhance the current landscape of implementers to stop dealing with these problems
part 1: the use of context
- if you have a topic, put it in
context. - if you see
context, use it for grouping.
- a: copy the context as-is. this means you intend to participate in the current topic.
- b: set your own context. this means you intend to participate in a new topic.
- c: set a null context. this means you are not participating in a topic, or you are unaware of
context.
- if you don’t see
context, do whatever.
- (possible compatibility for projects that don’t yet understand context: attach the original
contextif replying from a context-aware project, for example)
part 2: making context resolvable
- if it doesn’t resolve: just use it for grouping
- if it resolves: use its metadata as the “topic”
part 3: making context a Collection/OrderedCollection
-
orderedItemsshould contain all the “posts” in the “topic” - Add/Remove should be used to explicitly target the context.
- Add: treat it mostly the same as Create (transform it into a post).
- Remove: treat it mostly the same as Delete (get rid of your representation of the post).
if you aren’t aware of collections then ignore the target and just fall back to the logic in parts 1 and 2. that honestly gets us most of the way there, anyway. you have two paths to get to your desired property anyway: Add.object.context and Add.target. if this doesn’t resolve then it just degrades into a Create/Delete basically
part 4: there is no part 4
if you take away anything from this discussion, it should be that semantics matter. the clarity of those semantics also matters. let the logical grouping be the logical grouping, and the response be the response. let there be an actual context for doing contextual things.
the three-part description above should not be too hard to implement. there’s fallbacks at every step of the way to handle the unaware softwares. all i’m really asking is that people who are writing new code shouldn’t lock themselves into old assumptions, lest they take on technical debt before they’ve even begun, and lock the door to future implementers who want to do cooler things.
2 Likes
The problems you mention might exist, but they are so minor that no one else actually complained about them. And while that is the case I will rather spend my limited time to fix actual problems such as UX or documentation. Federation works good enough for now. Of course you are welcome to implement your suggestions in a real project, then I promise to have a look how it can federate with Lemmy.
Thank you so much for this very constructive discussion. This is exemplary of free software cooperation. Thank you all for building our software commons!
2 Likes
The further I go with the Discourse ActivityPub plugin, the more I think @trwnh is right about a few things, particularly the need to
- Make topics collections. The ActivityPub plugin already does this.
- Put the topic collection in the
@contextof objects in the topic.
The issue with just using inReplyTo becomes a bit more acute when you start to do things like bulk publishing and importing an Actor’s outbox (e.g. all of the activities in a Discourse category), which is what I’m currently working on. I haven’t yet done 2, but I suspect I will soon need to.
Actually, I forgot already added support for adding the collection in either the context or the target of an object (see here). Regarding having the object’s collection in the target this is to allow for supporting those following 400e.
See further
Personally, I think putting it in the context makes more sense (and is more spec compliant), but it’s easy enough to support both.
FEP-400e is going to be implemented in Streams:
This document specifies an ActivityPub compatible model for managing conversations as Collection objects with a common context.
Cool. The only slight issue I have with it is that I agree with those who have pointed out that it seems to use target inappropriately, see
or
I think context makes more sense, as @trwnh and others have advocated. I don’t really know how the FEP process works, but it seems like there’d be a decent case for substituting context for target.
cc @macgirvin
1 Like