šŸš€ ActivityPub Rocks Portal: Technical Discussion

Welcome :wave:

Here we discuss technical ins & outs required to publish and maintain the portal website.

Take note in discussions and:

:dart: Stay on-topic and try to formulate substantive contributions.

:warning: Posts that are off-topic will be forked to their own thread.

:speaking_head: Discuss anything, but focus on actionable follow-up.

:spiral_notepad: Use markdown formatting for clarity, like topic headers.

Above all ā€¦

:point_right:   Do not forget to update the ActivityPub Rocks Portal Wiki where needed !

The source of the current site is available at:

Technology Stacks

Incumbent: Haunt

https://dthompson.us/projects/haunt.html

Its used by the Guix website for instance (which I have looked under the hood).
(Like Christine Lemmer-Webber, Im a Lisper and in the Guix community)
(Its a very rare situation where Im ever the one with an incumbent stack :s )

Id prefer it if it gets stated explicitly what technical features are needed to justify moving off any stack
(as somebody who normally is the one suggesting alternative approaches Id expect this reciprocated).

Stay on Haunt until full consensus emerges.
Nothing is worse than things being chucked out while somebody is working on the next big thing.

Formats

Haunt has a couple of additional formats it can read:

  • sxml
  • html
  • texinfo
  • commonmark
  • skribe

Fwiw, Christine is using the Skribe format

i18n

This is pertinent given Guixā€™s own work with translatable content.

Heres a video regarding this:
https://10years.guix.gnu.org/program/#let-s-translate-guix-together-

If we stay on Haunt Id be happy to liaise with the developer community to ensure that the portal can handle multilingualism - as well as us pooling translatable content.
(the format PO comes to mind, not sure)

Their approach is to use a very GUI friendly translation tool.

And this is Guixā€™s i18n guide:

It is worth noting that Guixā€™s website is run using the SXML variant using a GNU orientated stack.
As such, it is operating with better technical solutions which may exclude some volunteers
(no doubt we need to focus on throughput).

Alternative: Vitejs/Astro

This tool for static sites has been very well recommended in multiple website builder threads recently:

I suspect that Astro will fulfill the criteria of most, though I would like to emphasize the benefits of minimal technology stacks.

i18n

This is Astroā€™s i18n guide:

Localisation is a win for me, and you have suggested something based upon Vite so I cant feel too aggrieved
(my nostrils were flaring thinking Lisp could still be in playā€¦)

General Points about i18n

One way or another we should feed off already translated content, which both silos can help with.

FWIW, its worth noticing that despite enough views, my appeal for widening diversity for the SH steering group we have come to nothing:

Perhaps there are more people out there who are keen on demonstrating the value of ActivityPub across different linguistic groupsā€¦

Forges

Although, no doubt Codebergā€™s page serving stack is low hanging fruit:
https://docs.codeberg.org/ci/
https://codeberg.page/

I did something equivalent for SourceHut - which was so simple it felt dastardly:
https://sr.ht/~sircmpwn/sourcehut/

IMHO, either Codeberg; Forgejo; or SourceHut are good initiatives.

Other (non HTML) Protocols

Any chance we can serve the ActivityPub Rocks Portalā€™s content for the protocol Gemini (in parallel)?

Once the technical structure is in place I dont mind doing the building for that.

Federated Technologies

In an ideal situation such a portal would be a living embodiment of federated technologies but I suspect this would require sustained commitment from multiple people for doing effectively.

1 Like

I think before talking technical options, one should first narrow down the features, one needs. If one wants to build a portal, see here, then one probably wants the following features:

  1. Automated build, deploy, etc ā€¦
  2. Automated recognition of stale links, e.g. a failing test
  3. To allow as many people to edit as possible use a simple format, e.g. Markdown
  4. To allow people to work together, the format should support good diffs in version control
  5. Finally, creating new pages should be as simple as possible, i.e. it should only require creating a file.
  6. Accessibility ā€¦ :question:
  7. Internationalization ā€¦ :question:
  8. Fediverse Integration ā€¦ :question:

:question: points mean: Someone else should write up some minimal requirements.

1 Like

Candidate static site generator: Astro

In follow-up to @indieterminacy and taking @helgeā€™s list into account, I do have a SSG preference, which is:

Vite is primarily a build tool, at a lower level than a dedicated static site generator. Astro is built with Vite, and provides a lot of features OOTB, like i18n and a11y, markdown + MDX support, and broken link detection. Forgejo is a project that uses it on Codeberg.

For automated builds we can use Codeberg CI to deploy on Codeberg Pages.

Fediverse integration is a :question: for this SSG, though, and probably isnā€™t offered rn. This also may be a point where, given requirements, Codeberg Pages isnā€™t able to offer the functionality. For simplicity sake I have a strong preference that Codeberg Pages is our publishing solution. Build & serve blazing-fast static pages.

1 Like

Accessible dev environment

I think that one additional requirement should be that coding for the website is in a common language, and using a stack with a large ecosystem. So that contributing to the portal code is low-barrier.

Vite/Astro with Typescript + popular-web-framework-of-choice fit that requirement, whereas Lisp, Guix, Scheme, etc. do not. There may be more suitable candidates.

1 Like

I reached out to the Guix-Devel website (given there are Haunt devs there):
https://mail.gnu.org/archive/html/guix-devel/2023-09/msg00449.html

I received this reply from David Thompson:

Iā€™m the creator and maintainer of Haunt. I canā€™t volunteer time to
maintain activitypub.rocks but I can help answer questions about Haunt
and help with bug fixes, patch review, etc. for Haunt itself. Haunt
posts can be written in Markdown, BTW. You donā€™t have to layer on the
obscurity by using a format like Skribe (even though Skribe is cool.)
I havenā€™t integrated anything i18n related in Haunt itself because I
donā€™t know a lot about it, but Guix being able to add in i18n on top
of what Haunt provides is a good sign that itā€™s workable. Iā€™m totally
open to include built-in i18n features in future releases.

I also understand and sympathize that keeping Haunt will probably be
swimming upstream against the pressure to be more mainstream. I guess
a positive spin on things is that ActivityPub has succeeded enough to
call attention to all the Scheme that was cleverly snuck in when
Christine was the driving force of the project. Itā€™s been a small
point of pride that activitypub.rocks was built with Haunt, so it
would be a bit disappointing to see it go away, but I understand that
whatever is easiest for the volunteers actually doing the work is the
right thing. Scheme is obviously better than TypeScript, though. :wink:

https://mail.gnu.org/archive/html/guix-devel/2023-09/msg00451.html

1 Like

History and Credits

As also mentioned on chat, given the requirement of an accessible dev environment (added to the wiki), I feel that as David already mentions, Haunt imho isnā€™t fulfilling that requirement.

But I think a proper way forward is to give credit where it is due, both to David and certainly also to @cwebber. A good way to do so is by adding a ā€œHistory of ActivityPub Rocksā€ section to the About page, that specifically mentions the way the site was constructed, tech used, and provide a link to the repository for those who are interested. I will update the ā€œThoughts on Landing Pageā€ I posted yesterday, to reflect that.

Internationalization (i18n)

I feel that - while support for i18n in the SSG is a MUST-have - maintaining actual translations is a nice-to-have. Whether language translations become available depends on community contributions. However, each language that is added is a serious increase to maintenance burden if the expectation is that they stay in sync with content changes in the primary language (English).

The approach taken depends on content strategy. It may well be that the portal only has few content, because it serves mainly to redirect people to appropriate places elsewhere on the web. Support for some of the more dominant languages is also an option, to reach good global coverage (think Spanish, Chinese, French, etc.)

1 Like

Should the Haunt setup be put to bed then indeed such a response would be appropriate.

I would extend it to ensure that the repository should be fully archived on Archive.org (perhaps both as a tar as well as an operable GitLab site).

For example, this is insufficient - as they do not provide content:

https://web.archive.org/web/2/https://gitlab.com/dustyweb/activitypub.rocks/-/blob/master/guix.scm
https://web.archive.org/web/2/https://gitlab.com/dustyweb/activitypub.rocks/-/issues/13

For a technology purporting to be of significant it feels like a major archival oversight.

Anybody got an account to do this?

1 Like

Another related question: We have this vibrant community here. It would be kind of cool, if one could include a ā€œcomponentā€ that displays recent posts on SocialHub.

Somebody with Discourse experience will probably know if such a component exists.

2 Likes

Actually itā€™s pretty easy with Discourse to provide a list of topics on a static site, or comments associated to a static page.

Another cool thing is that with Published Pages, we can host markdown-based pages here that can be edited by the community and have them at a stable URL like this one: https://socialhub.activitypub.rocks/pub/fediverse-enhancement-proposals#index

2 Likes

Iā€™ve played around with astro - starlight a bit. I donā€™t think it can be used for technical documentation without investing a lot of energy. I could neither figure out how to highlight lines of code or insert a ā€œcopy to clipboardā€ button. The instructions, I found only, all involved writing JavaScript and I wasnā€™t able to get it working (spending a limited amount of time).

However, I think the MDX format is what one wants here. From @how 's first link:

<script src="http://URL/javascripts/embed-topics.js"></script>
ā€¦
<d-topics-list discourse-url="URL" category="1234" per-page="5"></d-topics-list>

This sounds like something that can be turned into an MDX component. I expect similarly a Fediverse integration for Static Sites to come in the form of components.

I think my desire to have this type of integration pretty much forces tools from the JavaScript ecosystem on us. I donā€™t think using a tool that limits us to be able to do portal well instead of publishing technical documentation is a bad choice.

1 Like

I must say I would certainly prefer something minimalist that does not require JavaScript (i.e., degrades gracefully) and is mostly pre-built at compile time. We can also envision using API calls to Discourse and Codeberg to make things easier.

1 Like

Thereā€™s ready-made integrations for the code blocks with some of the functionality you mention. Used for instance on Astroā€™s docs, see example.

As mentioned this is taken care of in Astra. It does stuff compile-time and generates static output that is published to the site. You can try disabling JS of astro.build or stellar website and it doesnā€™t degrade. I added ā€œworks without JSā€ to requirements in the wiki.

2 Likes

I do not consider that to satisfy my requirement 3.:

This does not mean that we should not use Astro as a solution, but we should be aware of its limitations. Itā€™s not something that makes building technical documentation painless. Fortunately, we do not want to write technical documentation.

For multiple low-key edition of static pages, Iā€™d really recommend using Discourseā€™s Published Pages:

  1. Create a ā€˜web editorsā€™ group
  2. Create a dedicated category for the official site with create-reply-see permissions to the web editors, reply permissions to trusted people, and read-only to everyone else (only the first post makes it to the ā€˜published pageā€™)
  3. Make the first post (page) a wiki so that everyone in the community can amend the page, or moderate edition like: quote the first post and reply with suggested changes.

The process is transparent, as horizontal as we want it to be, and changes appear straight to the web without having to go through git deployment for selected pages (e.g., the Guide for new ActivityPub implementers works like this and appears at https://socialhub.activitypub.rocks/pub/guide-for-new-activitypub-implementers) ā€” one thing we might want to change is authorship: @nedjo receives notification for every change, and the page fails to reflect the multiple contributors.

You are right. Not the example I inlined. The other integration can be used in plain Markdown docs, but for highlighting and diffs will need something extra:

```sh ins={3,4} del={5,6}
  # Using NPM
  npx astro add @thewebforge/astro-code-blocks
  # Using Yarn
  yarn astro add @thewebforge/astro-code-blocks
  # Using PNPM
  pnpm astro add @thewebforge/astro-code-blocks
` ``

There may be good uses of this for selected sub-pages, but I do not think it can be a whole replacement of a static site generator and the workflow to maintain a site with that? So then this is additional functionality to take into account, once the ssg-based portal is underway.

Note thereā€™s a danger here. If the wiki isnā€™t the last in the thread no one is aware of edits, and anyone with permission can sneak in malicious URLā€™s and text.

Iā€™ve been thinking of the sort of tooling that could be used for this, and the idea we could use Fossil came to me.

  1. It supports easily creating and editing pages with Markdown.
  2. It obviously supports revision control, history, diffs, etc.
  3. It has a reasonable degree of a11y (Iā€™m blind and havenā€™t found obstacles using it).
  4. Does not require javascript.
  5. i18n is more of a question mark, not sure to what extent thatā€™s supported though one could probably do something with it.
  6. Very simple to deploy, replicate and archive.
  7. It gives us a lot of things for free, some of which we might of course not want (forum, real-time chat, issuesā€¦).

I realise itā€™s an unusual solution but I think it could suit this project.

Thank you, and welcome to SocialHub, @modulux. I like Fossil, but would be very hesitant to bring more tools/channels to the mix, as they tend to spread contributors thin and increase maintenance overhead non-linearly. Currently we have repositories under an organization account in the git-based Forgejo forge at https://codeberg.org/fediverse. For instance this is also the place where the FEP process has its home.

1 Like

Indeed Iā€™m promoting this system for selected pages that may be updated by more than just those who can compile the static site.

I think you get notifications of modifications on first-post wikis. Maybe itā€™s a host-only feature. But as this feature would only be available to an editors team, I see no danger here. People with no permission to edit the first post can still suggest changes by replying to the topic.

Thatā€™s an interesting proposition, and Iā€™ve been looking into it for a while, it remains something of a niche although I really appreciate the integration of issue tracking and so on within the system. But as @aschrijver mentioned, weā€™re kind of bound to Git these days. (I know some developers prefer Mercurial, but the vast majority adopted Git, and Codeberg primarily uses Git.)

As a matter of curiosity, how do you fare with Git with regard to blindness?
And do you have specific preferences for static site generators that are more accessible than others?

I think what we need is some way to spit out HTML pages and eventually grab live information at compile time from Forgejo (Codeberg) and Discourse (SocialHub) APIs to generate indices for FEPs, open issues and discussions, and so on. The process would mostly be automated and the site rebuilt whenever a change is made to either information sources, or on-demand, maybe via order in a Matrix room, or post to a bot on the Fediverse, or run manually. Many of the informational pages should be editable straight from the SocialHub to leave space for both automation and flexibility.

A quick look at the repositories on the Codeberg Fediverse shows that Hexo and Jekyll are used. I know everyone would like their own pet static site generator there, but we should focus on Markdown integration (since weā€™re using it everywhere)ā€¦ @aschrijver proposed to settle on Astro which seems to be a good option with native support for Markdown.