Decentralised Hashtag Search and Subscription Relay: Implementation Progress Report

I have recently finished my study project during which I started implementing Hash2Pub, so it is time for…

…another status update:

I have been working on Hash2Pub since March 2020 for a study project.
The main goals have been implementing a simplified post relay, its foundational DHT with load balancing, and then benchmark the effectivity and efficiency of that load balancing. While I had to focus on the academic useful aspects, this allowed me to lay some important groundwork

current implementation status

So currently my Hash2Pub implementation provides

  • an EpiChord DHT implementation in Haskell
    • its own UDP-based communication protocol
    • load balancing with k-choices (still untested)
  • a simplified RESTful relay service on top of the DHT
  • a testbed for performance evaluation
    • built with NixOS
    • utilising real-world input samples
    • system model, assumptions, szenarios

…well, that is almost true. Notice that I mention a testbed, but not the test/ simulation results.
Unfortunately there are still major bugs present preventing the simulation from running successfully in larger setups (> 20 nodes), so until these are found and fixed I don’t feel comfortable deriving any meaningful insights from this.

next up

The bad news: I won’t be able to work in this full-time anymore, making Hash2Pub my slower progressing side project now.

The good news: I can now work on things which are not interesting from an academic point of view, such as proper documentation, real-world use cases, and improving the code quality.

My roadmap for the near future is:

  • finding the bugs left that prevented the simulation from running properly
  • do some simulation runs
  • clean up code and improve module organisation
    • e.g by factoring out certain parts into separate modules (RingMap)
  • improve test coverage
  • improve code documentation
  • release public documentation of architecture, design decisions, and protocol
  • replace the simple HTTP relay by an ActivtyPub one

If you have ideas, comments, or want to join me working on this please contact me. The code base, but without the testbed setup and some documentation, is already publicly available. Big thanks to @hecate@pleroma.fr who already provided some help and PRs.

1 Like