FEP-d36d: Sharing Content Across Federated Forums

Standards Fediverse Enhancement Proposals
, ,
21

With 2100, if an instance goes down the community can still continue working based on a different instance

How? From my reading, 2100 is using the same Group to Group follow mechanism that d36d is proposing. I didn’t read anything in it that describes handling an instance going down. That sounds like a great feature and I’d love to merge it into d36d.

With d36d its nothing more than automated crossposts and will break if an instance goes down. So theres not much benefit to implementing it.

Basic threadiverse functionality breaks right now if an instance goes down. d36d doesn’t change that at all. The main purpose of it was to alleviate the duplicate thread issue where each instance has its own local community for a topic and users think they need to post to multiple communities. Discussion gets fragmented across these communities and users have trouble keeping up with where conversation is happening. This is a constant complaint I’ve seen across lemmy and kbin. I would argue that solving it would be a great benefit and if we can merge 2100 into d36d the benefit is even greater.

Such feedback isnt possible if there is no real-world implementation

You just gave invaluable feedback by pointing out your focus on community redundancy and 2100’s resiliency goal. Feedback before an implementation is vital to get to the point where implementation is worthwhile.

1 Like
22

How? From my reading, 2100 is using the same Group to Group follow mechanism that d36d is proposing. I didn’t read anything in it that describes handling an instance going down. That sounds like a great feature and I’d love to merge it into d36d.

Its mentioned in the summary, though there is no explanation how exactly it would work on the protocol level. It could definitely use more details and of course a working implementation.

Basic threadiverse functionality breaks right now if an instance goes down. d36d doesn’t change that at all. The main purpose of it was to alleviate the duplicate thread issue where each instance has its own local community for a topic and users think they need to post to multiple communities. Discussion gets fragmented across these communities and users have trouble keeping up with where conversation is happening. This is a constant complaint I’ve seen across lemmy and kbin. I would argue that solving it would be a great benefit and if we can merge 2100 into d36d the benefit is even greater.

There have been many such complaints right after the Reddit migration, but they calmed down a lot as people got used to the platform and existing ways to discover communities. Reddit also has plenty of duplicate threads across different communities. And to some extent thats expected because different communities will discuss the same topic differently. So I dont think this feature is really important, there are countless other issues on Lemmy which are much higher priority.

23

Hi, sorry for only replying now, I’ve been away for about a year now… I was really tired and had to take a break, and then I started my PhD… I’ll only be able to fit contributing to GNU social back into my schedule in the next month.

The third example scenario idea was (IIRC) that, if Group B implements fep-2100, and does not include a value for the gs:unbound attribute, then it means that Group B makes manual decisions (moderation) about whether to allow that “expansion” or not.

But if you look into our implementation of it, we haven’t included such moderation functionality so, for us, it is in fact (wrongly) dealt with as if scenario 1 (gnu-social/GroupSettings.php at v3 - gnu-social - Undefined Hackers: Git ).

The text where I stated that false and not present were the same is something written for a previous idea and wasn’t accordingly updated in a later revision… I guess I forgot to update that snippet as I never got to implement this scenario in GS v3.

1 Like
24

Regarding how it works if an instance goes down, we handle it as we handle everything else ActivityPub-related: it enters the failed queue with exponential backoff to retry later…

The manoeuvre of fep-2100, and why we claim that an unbounded group can still be used when, for example, the first instance of it is offline, is that when groups are unbound:true, they announce everything between them (as defined in fep-1b12), so they all work as mirrors, you can post to any of them, and that post will eventually show in all of them (as long as they are all linked both ways).

Of course the local instance should have its own version of every unbound group so it is fully independent of any other instance (N.B.: it doesn’t have to use the same nick!). !tech-general@A can be !tech-remote@B. This note was written in a previous revision of fep-2100, but at some point, we ended up removing it to maintain the spec flexible and light: we started by having too many specificities that were related to how things were done at GNU social, and we removed a lot trying to make it more general and simple. I will re-add some notes in the form of examples. If you wish to automate this behaviour of creating a local unbound group for every new interaction with a foreign unbound group, you could automatically drag the interaction to a locally linked unbound group or, if nonexistent, ask the user to create a new local unbound group linked to that one.

Finally, something I remember we spent some time wondering and thinking about was how to avoid too many unnecessary Announce requests around and we ended up agreeing that the amount here was manageable and that, in this context, the Forwarding from inbox mechanism would only bring up unnecessary trouble (thus the case for scenario 4). If you read the discussion here in SocialHub and in GNU social’s issues tracker, you’ll see that we started by considering other techniques such as temporarily moving points of authority or synchronising followers collections. But these were overly complicated strategies and were full of problems, really. The solution presented in fep-2100 is simple and doesn’t cause any significant overhead.

25

So, I was reading fep-d36d, and I don’t think fep-2100 should be merged with it.

fep-2100 is about linking entire groups (or organisations) for shared collaboration, or in the case of full unbound groups, so they are more resilient. It doesn’t go down to the objects being received in the group’s inboxes level (nor does it intend to).

I should add usage examples to fep-2100… You can have a group A that doesn’t want to receive activities from other groups, but other groups may be interested in receiving activities from group A.

For example, the group !tech@hackersatporto.com could follow a !tech-news@newspapper.com, and the other way around isn’t necessary.

An unbound group, however, could be !tech@fediverse.rocks and !tech@fediverse.cool, where both could behave as the same by having both linked each way (which will happen automatically if unbound:true on both).

If I understood fep-d36d correctly, it is “more aggressive”, in that it aims to deduplicate activities shared between different groups. fep-2100 doesn’t intend to do that - as we believe that a “!tech” group, just because it has a generic name, doesn’t have to be the same as every “!tech” group unless the author defines so in its settings - in which case we only deduplicate activities by IDs (as we already would do for every other ActivityPub activities).

Suppose, an instance may even have a !tech@A.com (for local !tech discussions and unbound:false) and a !tech-general (with unbound: true). And !tech-general will never receive the posts done to !tech@A.com, as it cannot follow it, so no dupes. Hence, if I want to interact with the cool posts from !tech@A.com, I’ll have to address them explicitly as no unbound group will include them.

Also, note that fep-2100 was designed inside the concept of federation adopted for GS v3 where most functionalities can be switched on and off and customised via plugins. We wanted this spec to be minimal, so implementers can decide the best functionalities that will make use of it on their platforms, as long as these few rules are followed, in terms of functionality and UI/UX, there are many possibilities :slight_smile:

1 Like
26

If I understood fep-d36d correctly, it is “more aggressive”, in that it aims to deduplicate activities shared between different groups. fep-2100 doesn’t intend to do that - as we believe that a “!tech” group, just because it has a generic name, doesn’t have to be the same as every “!tech” group unless the author defines so in its settings - in which case we only deduplicate activities by IDs (as we already would do for every other ActivityPub activities).
Suppose, an instance may even have a !tech@A.com (for local !tech discussions and unbound:false) and a !tech-general (with unbound: true). And !tech-general will never receive the posts done to !tech@A.com, as it cannot follow it, so no dupes. Hence, if I want to interact with the cool posts from !tech@A.com, I’ll have to address them explicitly as no unbound group will include them.

This doesn’t conflict with my intent behind fep-d36d at all. The name of the community is irrelevant to d36d. I wasn’t aware of the gs:unbound attribute but my proposal is basically the same as your when communities are gs:unbound: true.

In the case you mentioned, with !tech@A and !tech-general@B, since they’re not in a reciprocal relationship (they don’t follow each other) there would be no deduplication The deduping only occurs if two communities are in a reciprocal following relationship. If you have !tech-general@B (with gs:unbound: true) and !random@C (with gs:unbound: true) both following each other, according to both proposals they are now mirrors of each other. Under d36d, if an actor submits a post to !tech-general@B and then submits the same post to !random@C, the post to !random@C should be rejected because !random@C should already have received the Announce from !tech-general@B with that post.

1 Like
27

I see. Perhaps the tricky/confusing thing is the abusive use of the term “mirror”. In fep-2100 two groups only behave somewhat the same IFF they have the exact same linked_to and links_to collections, but they are still seen as distinct.

The example you provided, in the context of fep-2100, would be desired to not be deduplicated. I guess it’s a matter of how we interpret/understand a group. If that mechanism is desired in some platforms, maybe make fep-2100 a dependency/requirement of fep-d36d.

1 Like
28

@0x1C3B00DA If you want to write FEPs about Lemmy, I suggest you start by documenting the Activitypub extensions which are already being used. For example here or here.

You could also help by improving the Lemmy federation documentation, such as linking the FEPs which Lemmy implements (only nodeinfo and group federation so far I think).

2 Likes
29

Implementation deliberations from the side of mbin:

I see your problems now and get why just putting the mags together is not the solution you need. However I have a few problems with that proposition:

  1. It is a lot of work to implement and then test it.
  2. Since Lemmy does not implement it and most big communities are on Lemmy, us implementing it would have very little effect, since there are no big mBin instances, yet. So we would invest a lot of work for a, admittably nice, feature that will be of limited use because no one else is implementing it
  3. Right now most software will probably just answer with a “Accept” to a request to follow a group. Which will be a problem if the feature is more widespread, beacause it should be manual accept only
  4. What happens when instance A has magazine Y and Z from instance B, but magazine Y follows Z. Then a post to magazine Z will be send to magazine Y, too. Then we have a post that belongs to two magazines. I don’t know if our current system allows it and I am also not sure if it should allow it :smiley:
    So the options here are to
    • have the same post multiple times in the db which would waste space
    • just have the post once in the “original” magazine (which would kinda defeat the system in the first place, because it would then only work if an instance only follows magazine Y)
    • remodel the db design to allow a post to have relations to multiple magazines (which is eigher a lot of work or a huge amount of work, depending on how one implements it)

Point 1 and 4 are the biggest ones for me. Right now we just do not have that much dev capacity. We are pretty much just 3-5 devs with limited free time to work on mBin. Take me for example: I have on average one evening per week that I can spend with mBin.
If some one has a proposition to solve point 4 we can put this on the roadmap, but unless there is a good solution to that …

I think most people (or most devs at least) hoped that the biggest/best magazine/community with a specific topic would just drown the other ones out, but that doesn’t seem to happen since there are about 4 really big tech mgazines.

Point #4 is most pertinent to our spec discussion here. I wonder if @diogo or @0x1C3B00DA can think of a good solution to that case.

30

I responded on the github thread but I don’t fully understand the issue. The post would still only be owned by a single Group and Announced to following Groups. kbin/mbin support microblogging and if they store Announces by storing a new copy of it, then they’re already doing that so why is it different here? And the alternative is what’s happening now, which is a user will just create a new post in every community so you end up with the same amount of posts.

1 Like
31

@angus are you doing a version of group following here?

Is this an original feature of Discourse or were you following any prior art in the fediverse for this one?

32

Hey @erlend_sh, yes! The approach is primarily based on

It is also potentially compatible with

https://socialhub.activitypub.rocks/t/fep-400e-publicly-appendable-activitypub-collections

and

(and Mastodon’s yet-to-be-merged implementation based on the latter)

I’ll be writing up an overview of the implementation soon (and clarify what I mean by things like “potentially compatible” with FEP-400e), now that it’s a fully bi-directional ActivityPub server (e.g. categories on two Discourse instances can now follow each other)

3 Likes