Help improving federation between Lemmy and other projects

trwnh · January 20, 2023, 7:15am

it should be trivial to just extract the id from the inlined actor if you really must fetch it…

ideally, regardless of whether you used json or json-ld, you would be using a partial hydration strategy to expand the object (or graph) as needed. meaning, you would store nodes by their id and inject them into the json object (or the json-ld graph) as needed from your cache or via a network request.

example: you try to access actor.name but fail because that node has not been hydrated yet:

{
  "actor": "https://fed.brid.gy/snarfed.org",
  "object": "http://ds9.lemmy.ml/comment/1",
  "audience": "https://enterprise.lemmy.ml/c/tenforward",
  "type": "Like",
  "id": "https://fed.brid.gy/activities/like/fd61d070-7382-46a9-b2b7-6bb253732877"
}

so you try to expand the actor property:

{
  "actor": {
    "url": "https://fed.brid.gy/r/https://snarfed.org/",
    "image": {
      "url": "https://secure.gravatar.com/avatar/947b5f3f323da0ef785b6f02d9c265d6?s=96&d=blank&r=g",
      "type": "Image"
    },
    "type": "Person",
    "name": "Ryan Barrett",
    "icon": {
      "url": "https://secure.gravatar.com/avatar/947b5f3f323da0ef785b6f02d9c265d6?s=96&d=blank&r=g",
      "type": "Image"
    },
    "id": "https://fed.brid.gy/snarfed.org",
    "preferredUsername": "snarfed.org"
  },
  "object": "http://ds9.lemmy.ml/comment/1",
  "audience": "https://enterprise.lemmy.ml/c/tenforward",
  "type": "Like",
  "id": "https://fed.brid.gy/activities/like/fd61d070-7382-46a9-b2b7-6bb253732877"
}

now you can access actor.name successfully.

this way, if you are given an inline representation that you trust, you can just use it as-is. if you don’t trust it or it is not there, you can replace that node with a trusted version.

nutomic · January 20, 2023, 12:27pm

You are right this isnt complicated, in fact we are already doing the same thing in some other places. Unfortunately I dont have time to work on this, but if you want to make a contribution I would gladly guide you through the code.

WisTex · January 23, 2023, 9:28am

Integration with Hubzilla would be nice. Hubzilla already has discussion groups (which it calls public forums) so it already understands what a forum is. That would make federation between Lemmy and Hubzilla easier.

Right now Hubzilla’s interface more closely resembles a discussion group than a forum, but I am actually working on changing that by creating new views and themes that give it a more traditional forum interface.

The best way to federate would be to use Hubzilla’s native Zot6 protocol, but considering that most platforms use ActivityPub, you might get “more bang for the buck” by using that protocol instead. (It should be noted that ActivityPub is an addon in Hubzilla, so the site admin must activate it.)

An alternative would be to use OpenWebAuth, which would allow people to remotely authenticate with their existing social identity. If you integrate that technology, people from Lemmy and Hubzilla could remotely authenticate on each other’s website, and participate in each other’s forums with one social identity.

nutomic · January 24, 2023, 12:06pm

Activitypub is definitely the way to go, because other approaches would require major new features in Lemmy.

In order to get federation working, I need examples of the json data which Hubzilla sends, such as actors, objects and activities. These files will be integrated with our test suite to make sure they can be parsed successfully. See this folder for examples from other projects. If you could send me this data it should be easy to make both projects compatible.

WisTex · January 27, 2023, 12:46pm

I’ll see what I can find. I am not a developer on the Hubzilla project, but I’ll try to figure it out. Maybe @macgirvin and @Mario can shed better light on how all this works on the back end.

One thing that frustrates me is that Hubzilla had to format its ActivityPub messages in a way that is compatible with Mastodon. And since Mastodon does not currently understand what a forum or group is, a lot of creative hacks had to be made to get forums to work with Mastodon. (As someone developing themes for Hubzilla, this affects me since it changes how the data is stored in the database.)

For example, since Mastodon will reject messages from people you don’t follow, the first post in a forum thread must be owned by the forum, and not the actual author. If it is not done this way, Mastodon users who follow the forum will not see new posts.

This may be changing as Mastodon is considering adding groups. This would mean that forums on Lemmy and Hubzilla don’t have to pretend to be a regular social media account to work with Mastodon.

WisTex · January 27, 2023, 1:46pm

I still think OpenWebAuth is an important option to consider. It allows people to use their social identity as a log in. You could use one Lemmy account to log into any other Lemmy or Hubzilla or Streams or OpenWebAuth-compatible website. It allows people to participate directly on other websites, not just through their notifications on their own instance.

nutomic · January 28, 2023, 2:25pm

Ah I misunderstood. In that case it should be enough if you send me some links to different types of data on Hubzilla, specifically user and group profiles as well as different types of posts.

I commented here about Mastodon compatibility.

WisTex · January 29, 2023, 11:42am

Basically, there are four types of accounts: personal, public, community forum, and custom. Community forums behave slightly differently than the others.

I am not sure which forums have ActivityPub enabled, so I think the best thing to do would be to use a test forum. That way we can try all sorts of experiments without messing with a live channel.

If you want a local account, I can give you one. Eventually I will be opening up this website to anyone who wants to create an account, but the website is actually still at the pre-launch stage, so I limit who can sign up for now.

Here is an example of an ActivityPub-enabled public account (my account):

heluecht · January 30, 2023, 6:20am

On Lemmy (and Friendica and also gup.pe), group accounts are announcing the posts of their members. This works perfectly fine with a wide variety of implementations.

heluecht · January 30, 2023, 6:30am

Recently someone asked, why he can’t subscribe to a Friendica group from Lemmy. I made a short test and saw that Lemmy seems to try to fetch the system actor from the root directory. Is this all or is there some more that Lemmy does to be able to subscribe to a group on another server?

nutomic · January 30, 2023, 2:52pm

That should work in general, for example I am able to subscribe to https://forum.friendi.ca/profile/helpers from lemmy.ml without any problems. The system actor you mention is completely optional, its no problem if that returns an error. So if there is a problem I need more details.

nutomic · January 30, 2023, 3:24pm

I added some tests for Hubzilla data to Lemmy and they are passing without any code changes. So I suggest you try federating with an actual Lemmy instance like ds9.lemmy.ml and see if it works. One problem I noticed is that Hubzilla groups create posts directly instead of using announce, which means that content in Hubzilla groups cant be browsed from Lemmy. The other way might still work.

heluecht · January 30, 2023, 6:21pm

Ah okay. Possibly there is a problem with the server that I tested with: lemmy.schuerz.at and I tried to subscribe to my development server: test1@pirati.ca

Which server do you recommend for having a test account to test all this stuff?

nutomic · January 31, 2023, 1:10pm

You can use ds9.lemmy.ml or enterprise.lemmy.ml which are automatically upgraded to latest dev versions. Let me know when you register as I need to approve the account.

informapirata · February 6, 2023, 4:24pm

There is only one GUP instance, even if there is the following GUP account which is used as a test account: @testgroup@a.gup.pe

heluecht · February 7, 2023, 2:38pm

I yesterday returned from the FOSDEM, so today I can continue my work. I registered now as “heluecht” on DS9.

nutomic · February 7, 2023, 3:27pm

I approved your account.

KelsonV · February 13, 2023, 4:47am

I’ve been testing GoToSocial with Lemmy. As of 0.7.0rc2:

GTS can:

Fetch Lemmy groups, users, posts, and comments.
Like, boost or reply to Lemmy posts or comments
Federate those replies and boosts to GTS or Mastodon followers.

GTS can’t:

Follow Lemmy groups or users
Federate those likes or replies back to Lemmy.

GoToSocial is stricter than most ActivityPub implementations about HTTP signatures, and I am seeing “http request wasn’t signed or http signature was invalid” in the debug logs when Lemmy responds to a follow request [edit] and to a reply.

In case that’s not the issue, or not the only issue, here’s a sample of the JSON being sent for following:

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "actor": "https://gts.keysmash.xyz/users/kelsonv",
  "id": "https://gts.keysmash.xyz/users/kelsonv/follow/01R19THB79WY9H1AQQYVRGVPCT",
  "object": "https://lemmy.ml/c/fediverse",
  "to": "https://lemmy.ml/c/fediverse",
  "type": "Follow"
}

Edit: and here’s a sample of what GTS is building for the reply:

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "actor": "https://gts.keysmash.xyz/users/kelsonv",
  "cc": [
    "https://gts.keysmash.xyz/users/kelsonv/followers",
    "https://lemmy.ml/u/humanetech"
  ],
  "id": "https://gts.keysmash.xyz/users/kelsonv/statuses/01GS4HQM3X5PCE182TENA24ASE/activity",
  "object": {
    "attachment": [],
    "attributedTo": "https://gts.keysmash.xyz/users/kelsonv",
    "cc": [
      "https://gts.keysmash.xyz/users/kelsonv/followers",
      "https://lemmy.ml/u/humanetech"
    ],
    "content": "\\u003cp\\u003e\\u003cspan class=\"h-card\"\\u003e\\u003ca href=\"https://lemmy.ml/u/humanetech\" class=\"u-url mention\" rel=\"nofollow noreferrer noopener\" target=\"_blank\"\\u003e@\\u003cspan\\u003ehumanetech\\u003c/span\\u003e\\u003c/a\\u003e\\u003c/span\\u003e I just signed up over at SocialHub, and I'm reporting some federation issues with GoToSocial and Lemmy.\\u003c/p\\u003e",
    "id": "https://gts.keysmash.xyz/users/kelsonv/statuses/01GS4HQM3X5PCE182TENA24ASE",
    "inReplyTo": "https://lemmy.ml/post/746045",
    "published": "2023-02-13T04:50:05Z",
    "replies": {
      "first": {
        "id": "https://gts.keysmash.xyz/users/kelsonv/statuses/01GS4HQM3X5PCE182TENA24ASE/replies?page=true",
        "next": "https://gts.keysmash.xyz/users/kelsonv/statuses/01GS4HQM3X5PCE182TENA24ASE/replies?only_other_accounts=false\\u0026page=true",
        "partOf": "https://gts.keysmash.xyz/users/kelsonv/statuses/01GS4HQM3X5PCE182TENA24ASE/replies",
        "type": "CollectionPage"
      },
      "id": "https://gts.keysmash.xyz/users/kelsonv/statuses/01GS4HQM3X5PCE182TENA24ASE/replies",
      "type": "Collection"
    },
    "sensitive": false,
    "summary": "Please promote your SocialHub. It is important!",
    "tag": {
      "href": "https://lemmy.ml/u/humanetech",
      "name": "@humanetech@lemmy.ml",
      "type": "Mention"
    },
    "to": "https://www.w3.org/ns/activitystreams#Public",
    "type": "Note",
    "url": "https://gts.keysmash.xyz/@kelsonv/statuses/01GS4HQM3X5PCE182TENA24ASE"
  },
  "published": "2023-02-13T04:50:05Z",
  "to": "https://www.w3.org/ns/activitystreams#Public",
  "type": "Create"
}

nutomic · February 13, 2023, 2:15pm

Based on the GoToSocial documentation, it makes signed fetch mandatory. This functionality is not yet implemented in Lemmy, so if this really cant be disabled in GTS, then its impossible for the two projects to federate.