I'm building fediverse.to!

Hi all! My name is Ayush, and I’m building fediverse.to to help people navigate the fediverse more effectively. If you have some time to spare, I could really use some early feedback and ideas for the future.

Full announcement here: https://www.buymeacoffee.com/ayushsharma/announcing-fediverse

Thanks,
Ayush.

2 Likes

It’s “only” about mastodon instances?

Nope, there’s instances from all over the fediverse, including Pleroma, Misskey, etc. I don’t know which software each instance is running since I’m not polling/scraping that info, but it’s on the feature list.

1 Like

is it a crawler? what are you crawling for ?

Currently I’m fetching the metadata API for each instance. It gives me user stats, language, description, etc. Categories I fetch from the instances.social API plus pattern searches in Python.

Been doing this for over 10 years, we have an API etc if you need any easy ways to populate all servers etc. https://fediverse.observer

2 Likes

This one ?

Your API was already on my list. Thanks for building it!

That’s right. Also using the API on mastodon.social.

Hi all :wave:. I have an update on fediverse.to. I’m now tracking 84 other fediverse software which are all searchable.

I need some advice on how to categorize instances.

I’m currently using a list of limited keywords to group instances into categories, but the keywords list is very limited and based on the categories that instances.social used to return in the API. I’d like to improve this feature by adding more keywords and categories but I’m not sure how to do that. Is there a list somewhere on general categories and keywords that I can use? Is there a smarter way to go about this?

Any and all advice is appreciated. Thanks :slight_smile:

1 Like

FWIW, we just removed all of our tracker support. Some of us came here to get away from being the product. We’re just people communicating with other people and it’s frankly nobody else’s business what software we use or what we like to talk about. Anyway, not a problem. Don’t come looking for us because you won’t find us, and if you do, we’ll be sure to prevent it from happening again.

1 Like

Once I got annoyed when I found out that over half the traffic to my single-user dev site was Friendica instances trying to figure out what kind of software I was running by repeatedly probing for APIs and interfaces from a huge list of those they knew about. I ended up writing a plugin to redirect all of these requests back to Michael Vogel’s site. I suppose the moral of this story is that if a site isn’t running Nodeinfo, just leave them alone.

1 Like

For example we display the server system of a contact to show the diversity of the Fediverse. Also not all group implementations are done in the same way, so we might end up in having to implement target specific content. (By now we are excluding Peertube groups from our forum list, since these groups aren’t groups in the way other implementations are using them). Also we might in the future implement some Lemmy specific stuff (like their spoilers). For these things we need to know the remote system.

Since not all systems (and versions) support Nodeinfo, we have to query stuff like the old statistics.json and some implementation specific endpoints. Especially when those endpoints don’t exist on the system, they shouldn’t create a measurable load on a system.

However, in the future we could implement a mechanism that queries the robots.txt first and looks specifically for some “friendica” entry there.

display the server system of a contact to show the diversity of the Fediverse

ack, it is also important cause of “compatibility” layers for software not following the Conformance Section (not supporting “the entirety”). Personally my software would like to know what it can send:
ActivityPub or only Note or only Banana …
The standard has a generator field.

All:
In the last 4 meetings, Feb.-May, we talked about groups.
The parties who attend the meetings will do it together and interoperable.
The next meeting Developers Meeting 7 : A standard way to query in ActivityPub and we can also talk about “Potential Actions” and how to self-describe workflows.

Yeah, we are providing information that generator field.

First time I hear about statistics.json. Do you have a link with more info on it?

(OT: This ever growing list of incompatibilities to take into account seem to me unsustainable in long-term, and true interop being on a slow ‘death march’ if we don’t find ways to improve our practices)

1 Like

The statistics.json had been introduced by Diaspora. They replaced it with the Nodeinfo, but some older systems still exist that use this endpoint.

BTW: I don’t think that the incompatibilities are growing. Most detection problems occur with older system.

1 Like

It wasn’t one Friendica site hitting some dead endpoints every week or two. It was every Friendica site checking the same 10-20 endpoints over and over and over again - every few seconds; all day, every day 24/7/365. To be fair, this was a year or two ago and I haven’t observed this anti-social behaviour recently.

Nowadays these sites are much more difficult to categorise. Every site determines their own software project and brand and they are much more resistant to data harvesting.