I’ve spent the last couple of days setting up a local, dockerized, minimal config sandbox. It can host production releases of fediverse software and automatically provision the DNS and SSL certificates necessary to allow them to federate with each other unmodified, or nearly so. This includes being able to federate to and from a service that’s running on the docker host. The goal is to make it easy to test and debug federation issues in a controlled, non-production environment.
I’m calling it Sandcastles.
At the moment I’ve only provided a configuration to run Mastodon, because it’s the only released application I personally have any familiarity with. I will certainly add a Letterbook config when the project is a little more mature. And I will probably extend that to other applications over time, but I would also be thrilled if other people contributed configurations. I think that could be a significant boost to fedi developers, and it’s something that sys admins would be very well positioned to do, without needing any real development experience.
I also know that @helge built something similar with fedi-pasture. (I think that’s the one?) The diference here is that there’s no need to patch and rebuild other applications from source to remove security requirements like HTTPS and domain names, rather than IP addresses. Instead that’s all provided as infrastructure, with only a minor change to add a local trusted certificate authority.
Would be great to have mastadon and https://rdf-pub.org/ running on my local machine to start implementing the integration. until no wi was only implementing the c2s part, but i would like to satrt federation soon.
Yes, if you set the mastodon.castle entry in your hosts file, you should be able to open it in your browser. If you’re running a service locally on your host machine (port 5127, but you can change that), mastodon should be able to reach it via the host.castle domain.
What this sandbox is providing is automatic SSL connections at those domain names. And also DNS for those domain names. That should remove some barriers to running other fediverse servers locally so you can experiment with federation issues. Those servers very often require you to have properly configured https and dns, which can be hard to setup and manage without just doing it live on the internet.
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3456dbfe43c6 smallstep/step-ca:0.25.0 "/bin/bash /entrypoi…" 5 minutes ago Up 5 minutes (healthy) sandcastles-root-ca-run-3a868af35e41
3f2309aa160a bitnami/mastodon:4 "/opt/bitnami/script…" About an hour ago Up 5 minutes sandcastles-mastodon-streaming-1
3c077bbd86d9 bitnami/mastodon:4 "/opt/bitnami/script…" About an hour ago Up 5 minutes sandcastles-mastodon-sidekiq-1
fa4a2ab16f10 mastodon-sandcastle "/opt/bitnami/script…" About an hour ago Up 5 minutes 3000/tcp mastodon.castle
0ce7b8e869e0 traefik-sandcastle "/entrypoint.sh trae…" About an hour ago Up 5 minutes 0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp dashboard.castle
90a1c12231b6 bitnami/elasticsearch:8 "/opt/bitnami/script…" About an hour ago Up 5 minutes sandcastles-mastodon_es-1
f86f0284f881 bitnami/postgresql:16 "/opt/bitnami/script…" About an hour ago Up 5 minutes sandcastles-mastodon_db-1
0955ad34a22c bitnami/redis:7.0 "/opt/bitnami/script…" About an hour ago Up 5 minutes sandcastles-mastodon_redis-1
curl --location --request GET 'https://mastodon.castle'
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
docker compose run root-ca
WARN[0000] Found orphan containers ([sandcastles-mastodon-streaming-1 sandcastles-mastodon-sidekiq-1 mastodon.castle sandcastles-mastodon_es-1 sandcastles-mastodon_db-1 sandcastles-mastodon_redis-1]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
badger 2023/11/13 18:19:19 INFO: All 1 tables opened in 1ms
badger 2023/11/13 18:19:19 INFO: Replaying file id: 0 at offset: 45392
badger 2023/11/13 18:19:19 INFO: Replay took: 13.686µs
2023/11/13 18:19:19 Starting Smallstep CA/0.25.0 (linux/amd64)
2023/11/13 18:19:19 Documentation: https://u.step.sm/docs/ca
2023/11/13 18:19:19 Community Discord: https://u.step.sm/discord
2023/11/13 18:19:19 Config file: /home/step/config/ca.json
2023/11/13 18:19:19 The primary server URL is https://root-ca.castle:9000
2023/11/13 18:19:19 Root certificates are available at https://root-ca.castle:9000/roots.pem
2023/11/13 18:19:19 Additional configured hostnames: root-ca, localhost
2023/11/13 18:19:19 X.509 Root Fingerprint: 1ea743b5710430e1d4e8d35fe047f80da23ea80b442bb7373a57492f4caebfeb
2023/11/13 18:19:19 Serving HTTPS on :9000 ...
INFO[0030] duration="62.778µs" duration-ns=62778 fields.time="2023-11-13T18:19:49Z" method=GET name=ca path=/health protocol=HTTP/2.0 referer= remote-address=127.0.0.1 request-id=cl96hha0fuec73et02lg size=16 status=200 user-agent="Smallstep CLI/0.25.0 (linux/amd64)" user-id=
INFO[0060] duration="30.918µs" duration-ns=30918 fields.time="2023-11-13T18:20:19Z" method=GET name=ca path=/health protocol=HTTP/2.0 referer= remote-address=127.0.0.1 request-id=cl96hoq0fuec73et02m0 size=16 status=200 user-agent="Smallstep CLI/0.25.0 (linux/amd64)" user-id=
Yes, that’s because your host computer doesn’t trust the certificate. You can configure your computer to trust it, which is what’s going on in step 5. Did you run that trust.bash script?
If you don’t want to do that system-wide, then it’s theoretically possible to mark the certificate as trusted, or skip certificate chain validation, in each program and library that you want to use. But the way you would do that is specific to each of those programs.
For instance, curl has the --with-ca-bundle flag. You can use the step-cli to download the ca bundle from root-ca.castle. Then you would pass that bundle to curl every time you want to make a request to a service running in this sandbox.
it’s running now and i can access it via browser.
But i didn’t get the confirmation email if i create a login.
I’ve to check, haw i can login as admin. But today there is no time left ;-(
The mastodon config is taken from one that bitnami maintains. There’s a comment in the mastodon.castle.yml file with the admin password.
That’s a good idea about email. The sandcastle services obviously can’t/shouldn’t send email to real addresses, but it should be possible to integrate maildev into the sandbox.
how can i setup an test account, if i cannot receive emails?
Would you like to do something like this for rdf-pub ? There is already a docker image and also some kind of docker-compose incl. keycloak and mailhog. “some kind of” in this case means i have to check if it still works
The pre-configured mastodon instance already has an admin account. If you create a test account for rdfpub, I don’t know? I assume that’s something you can do.
I’d be happy to accept patches that add a preconfigured rdfpub instance. I’m not sure when I would be able to make time to add it myself, especially as I don’t know much of anything about the project, or how to run it. On a long enough timeline, I would eventually get to it, but you’re better off sharing that config yourself if it’s something you want to have included.
I think we need to combine our know-how I’ll do some tests with the rdf-pub docker stuff and get back to you. I don’t know yet when I’ll have time to work on it either. i’ll try to find time.
Basically, an integration of keycloack would be interesting, since Activity-pub C2S is based on Oauth2.
The “old” docker-compose is here: docker-compose.yml It’s still working, but the rdf-pub version is old.
Good stuff! I was able to get both this and fediverse-pasture up and running without very much trouble, and it’s encouraging to see things moving in a very sensible-looking direction.
If adding stuff to /etc/hosts isn’t practical, an alternative is to add a forward proxy to the docker-compose configuration:
Then set up a new browser profile configured to use it. This works for both fediverse-pasture and Sandcastles. Maybe it’s possible to configure Traefik to do this directly; I took a quick look but didn’t feel like reading through all their config documentation
Do you and @helge plan to merge this with fediverse-pasture somehow? Two projects, each with its own collection of Docker configurations, doesn’t seem like an ideal situation.
Feature request: I think that being able to start multiple instances of the same software, eg “mastodon1.castle” and “mastodon2.castle”, perhaps with slightly different configurations, could come in very useful.
Ok, i have now taken some time and created a new release. It contains the updated docker.compose.yml and updated instructions for use after docker-compose up
How can we go foreward with a developer who has no knowlegde about reverse-proxy, CA …
Next steps with rdf-pub is the S2S federation, so it would be really create to have a setup with mastodon on a local machine.
how can i setup an test account, if i cannot receive emails?
