test.activitypub.rocks has been down for a long time, as pretty much everyone knows. More or less it’s server issues, and I can be blamed since I run the server (and wrote the test suite).
Why hasn’t it gone back up though? Mainly because I’m a) very busy b) know how poor the test suite is at fulfilling the needs of the community and that makes it hard to feel motivated to put in the work.
Here are some quirks about test.activitypub.rocks:
- It was done in a rush to meet the standards deadline. The main purpose was to collect implementation reports, and given the rush some things are funny about it.
- It was done in such a rush that it’s a completely separate program that’s just bundled straight into the same codebase as the test activitypub implementation I was writing, Pubstrate (which has some good ideas but which I won’t recommend either).
- Technically it’s two test suites. Scratch that, it’s a test suite and a questionaire:
- There’s a test suite for the client to server protocol, which really does resemble a test suite; it makes many requests against your server and sees if they work right. (It’s buggy, though.)
- There’s a mostly-questionaire (but it runs a couple tests, I think) for the server-to-server protocol. Why? Because a) performing a test suite on an asynchronous protocol is already hard enough (when does the test complete? it can be done with timeouts, but it’s gross) and b) many of the requirements required that some authentication or authorization take place, but due to process reasons we couldn’t specify what that was, and c) at that point we already had an unusual amount of interop, so we decided that if multiple implementations could federate on a feature and could confirm that in the questionaire, we could consider that sufficient. (Even weirder, some of the questions would involve the server doing something to your server, but then still having a question for the user to confirm if the thing worked.)
- A proper test suite wouldn’t work like this. It would be fully automated.
It does look pretty though, that’s the only thing I’ll give it credit for. 
Where to from here? There are really a few options:
- Do nothing, things remain broken. Obviously not ideal!
- I get back up the test suite. Again, it’s hard for me to get motivated about this but I could probably do it. But then I fear once I get it up all I’ll get in response is THIS is the test suite? WTF, this isn’t a test suite at all!
- Someone else could get up the test suite based on the current codebase and I could transfer running it to them. Notably someone tried this already but they weren’t familiar with Guix, and that’s currently the easiest way to get it up and running
- Someone else could write a new, much better version of the test suite. In fact we discussed this on a recent SocialCG call and I was asked how I would feel about that, and I more or less said I thought that would be great. But someone needs to do it. Who? Maybe it’s you!
So… what next?
discussions and decision making.