United Software Development: A new paradigm?

CoderGlue: Let’s Code Together :wink:

This topic was inspired by FedeProxy project, and my interactions on their forum. There’s a wink in the title, because “CoderGlue” name and icon design is just a proposal I made, as alternative more descriptive project name. Nothing more than that.

Culture and process

I really like how the team is setting up the FedeProxy project organisation. A thorough approach, deep thought, and proper preparation. So too on diversity:

@dachary: Diversity: First action

I’d like to propose that a first action is done to improve the diversity in the fedeproxy community. It is still tiny and it should not be too difficult :slight_smile: Two ideas:

Referring to two other topics on FedeProxy forum and knitting things together:

There’s some great feedback on the toot already, right? Loïc, you’ve got your audience right there on the fedi :blush:
Also we’ve got diversity in abundunce in fedi culture.

At SocialHub we just presented in our first “ActivityPub for Administrations” event the slogan and rallying cry “Social Networking Reimagined”, and described fedi culture as “United in Diversity”.

These two things can be embraced by FedeProxy, and become deeply engrained in your approach. First of all for your own project development. Here you can…

Dogfood your own development process that you’ll learn along the way, while dogfooding the fediverse.

And incorporate that in your ‘product experience’ i.e. as product features. I think your ‘proposition’ can be strongest the more you think about the Domain, and the less about the technical reality of what the software does (i.e. a bridge / proxy between code forges). In software development things all starts with culture and process.

I’ve worked with (scaled) agile in the past, as tech lead and product owner in quite large (200+ devs, 100+ consultants), quite diverse (60+ natiionalities) environment, where we adopted a Spotify Engineering Culture. I have also seen during my whole tech career (which has been mostly FOSS, fedi, humanetech advocacy & community volunteering for the past couple of years… not much ‘career’ in that. It’s all humble pie :wink: ) plenty of times where processes, culture and communication are the bottlenecks to successful projects. Not the tech itself.

Here can be the absolute strong points of FedeProxy, that might make it a killer app! By thinking holistically about what the objective is, and go from there:

Facilitating great, open to anyone, community-driven, successful free software projects that can sustain themselves.

Now Spotify Engineering Culture is interesting. You can go for something similar, yet entirely different at the same time. Imagine if FedeProxy could deliver their own vision to open software development culture + processes with videos similar to these (look through the specifics, those are unimportant):

(and here’s the video of Part 1, more focused on Scrum / agile)

Discover the Domain

Now, is it worthwhile to go the extra mile and explore the above? It will certainly consume part of your funding in extra time spent.

But what you have to do anyway is to analyse and elaborate the application / ‘business’ domain where you want to reach your project objectives. Right now, FedeProxy is conducting User Research. Very good and utterly important to gather as much feedback as possible.

But there’s a possible pitfall too. What users are you researching? For what domain? Currently on the forum this occurs in the “UX / User research” category. But UX is a step too far, if you do not know the domain. There’s a risk to start thinking about concrete features and functionality, UI widgets even, way too fast.

Note that there’s no need to search for a new paradigm - which I have named “United Software Development” - in a full deep-dive either. Just determine if it can be in your Vision! Then from there you can work backwards, find stuff that may be Roadmap items, skin further… until you drill-down to a MVP.

I feel that with a proper approach, FedeProxy may not be alone in chasing the paradigm. A host of other projects might spring up. This is the reason why I posted to #fediversity:fediverse-futures instead of on the FedeProxy forum.

The “United Software Development” paradigm

To conclude this brainstorm topic, I’ll try to come up with a first definition of the new paradigm that I feel tingling below the surface, just waiting to be uncovered…

United Software Development

“Free, libre software development, processes and culture united,. Sustainable, open and accessible to anyone.”

Open source software has eaten the world. Free software is still struggling. FOSS maintainers have a very hard time to manage their projects, let alone sustain their own life with a decent income.

United software development leverages the decentralized web, and combines all the best-practices and lessons-learned in the past decades to shape a way forwards. To facilitate a collective crafting of humane technology where people are in control, and that is to the benefit of humanity.

1 Like

and the (non-technical) communities they’re working for. If software is developed for developers, then it’s useless and a waste of time. Engineering culture is way too self-serving and mostly uninterested in what is called real life. Well, if you cannot feed yourself nor have decent health care, nor the freedom to celebrate your own culture, no software is going to make a difference. What about this?

1 Like

Back in January this year, while drafting the fedeproxy project description, I was tempted to go one step further. Instead of pushing for forge federation, why not advocate for what naturally follows: federated development?

I do not, however, have your talent for expressing these ideas and decided to aim for “forge federation”, a simpler idea that I was able to articulate and an enabler for what you describe as United Software Development. When you write leverages the decentralized web, I feel this is what I desire but it is blurry.

Here is a practical problem I think it would solve. As a developer I sometime have strong opinions on how I want to do things. I’m not a dictator on my own projects nor am I a pest when contributing to other Free Software projects. Yet I sometime drive people away because they disagree with how I do things. And I sometime run away from projects because I disagree with the choices that are made. In other words there does not seem to be any alternative: you are either in the project or out. It is not just about centralized forges (although it does not help, of course), it is a mindset.

I would like to live in a world where I could modify an aspect of a Free Software but not fork it. A federated modification would be a change that only exist for yourself (just like a fork) but that is linked to where it comes from. It would get activity notifications from where it comes from, giving it a chance to notice when and if it would conflict because the code diverged. The upstream it comes from could also be notified that there exists modifications in the fediverse on this particular piece of code, giving it a chance to discover interesting contributions that may have not been contributed back just yet. And that would not just be about code: issues, discussions, roadmaps, releases, CI configuration: every bit that keeps a software flowing could be similarly federated.

It would be living a dream :slight_smile:

1 Like

I love both @how’s and your response. It makes very clear the breath and scope, but also the complexity that inherently comes with it. There’s so many choices to make in terms of processess, techstack, organization and project governance, as well as how you involve your stakeholders and ensure you’ll deliver what they want and need.

(Note: I am going a bit wide in my response, so please bear with me or skip sections…)

Fediverse Futures

All-in-all I think this goes way beyond what a project like FedeProxy can deliver. For the paradigm to be meaningfully realized, an ecosystem of projects should dedicate to the vision and gradually fill in parts and gaps like a quilt. And it would help if these were more task-oriented in their design (see: From silo-first to task-oriented federated app design).

The paradigm of “United Software Development” itself, would stand on its own as an independent initiative. One that anyone can embrace, and that explores ideas, visions, concepts, etc. where people may jump in to elaborate in whatever way.

Note that this is the reason why I set up Fediverse Futures. Because the way of thinking (out-of-the-box) and approach of exploring what “leveraging the decentralized web” means, is something that individual FOSS projects cannot address by themselves. And it requires strong community, with - I am hoping - SocialHub playing a key role.

Without such efforts, we get a “watered down cup of federated tea” with, as I described elsewhere, approaches such as:

  • I want forum → I develop forum → How can forum federate?
  • I want video → I develop video → How can video federate?

I am thinking of ways to take this initiative next-level, beyond TL;DR forum topics and with a separate non-technical companion Fediverse Futures brainstorming area on Lemmy. I feel this should be part of Organizing for SocialHub Community Empowerment.

Sustainable communities

Part of @how’s addendum to United Software Development rightfully refers to both the sustainability of the FOSS projects as well as the communities they serve. It includes the economics, the innovative and new business and revenue models that society needs to progress to a bright future.

This is a subject very dear to my heart as well, and one I intend to help explore in all aspects (note that all my work as humanetech, FOSS and fedi advocate, makes me poorer and poorer, spending savings I made from a previous IT career. It is NOT sustainable in the long run, atm).

One Fediverse Futures topic that ‘Sustainable Communities’ hinges on, say the groundwork, is another paradigm:

Forget the technical aspects for a bit. What this paradigm should solve is that there’s a gazillion communities, all doing much-needed positive things, but doing them in a very fragmented landscape without much awareness of complementary or overlapping initiatives. This way we are missing out on the synergy, or outright reinventing wheels.

“Community Has No Boundary” then is first and foremost a binding force and communication medium to bring all this closer together. The decentralized web is an excellent foundation to build this upon.

Then, when there’s a more solid concept of what Community entails, we come to how they interact in a social fabric. And now we get to different domains to be modeled on top of community concepts. E.g. we get to Governance. Note again that the domain exsists regardless of technical implementation, and that - depending on context and situation - different governance models are applicable. They have different scopes too… governance within communities and across communities, for instance.

Not easy to model, but just like ActivityStreams did for “Social Interaction” there could be ‘starter models’ that fill in the primitives. With Governance there might be e.g. Governance Policy where there’s an endless variation of policies, but the semantic model to define them need not be specified in detail from the start.

@hamishcampbell is having solid ideas and exploring What would a fediverse "governance" body look like?

While both Community and Governance are independent domains wrt FedeProxy’s core domain, it should be clear that the project may use them to model on top of. And that doing so yields benefits, as the domains - when expressed as Linked Data vocabularies and AP extensions - allows others to independently extend/expand the ecosystem.

(Nothing new here… this is the whole idea of AP being a Linked Data spec, but on the whole this aspect is under-utilized and much of its potential remains untapped)

Federated Development

@dachary my talents aren’t that great either :grin:   I am just zooming out to a very high level, thinking strategically about a future Fediverse, and then hand some - hopefully - inspiring ideas, hoping they’ll trigger others to jump in and brainstorm wildly with me, and elaborate together. You just did that in a fabulous way.

The idea you just outlined is deeply inspiring on its own, and fits right under the umbrella of United Software Development. It combines with the Community + Governance addressed above.

I think what it boils down to is that the concept of a Project already constitutes sort of a silo, that - in many cases - hampers the potential unity of people involved and using it.

In the current workflows of git and online code forges a lot of collaboration concepts are already present, but they might be taken a way further than they currently are. A Fork sort of means “You have the freedom to do it your own way”, but in doing so (if it is beyond the purpose of creating a PR) you effectively spin up your own Project, a new silo, with a promise of “Maybe we get together later or at least still benefit from each other”. But this is implicit. Note, Gogs and Gitea are good example of this.

The federated development, part of United Software Development, would mean this boxed-in concept of Projects fades. Instead of loose collection of Projects that may or may not cooperate, you get to the idea of a Development Process that starts along certain Objectives, and then becomes a ‘living thing’… it organically branches out across the decentralized web driven by anyone’s incentive to add to it.

It is a great dream, and should be part of the United Software Development vision.

@dachary in Fedeproxy Updates, April 2021 - Events - Fedeproxy :

Create a CoderGlue infrastructure in addition to fedeproxy to host another instance of the project. And experiment with them having a federated life together. At the beginning it will be a little difficult because … well because fedeproxy does not exist yet and everything has to be done manually. But it is going to be an ideal ground for experimentation.

This is a nice idea and good way to explore what federated development means. But I was thinking something else, triggered by reading this…

Namely the possible clean breakdown of the “Unified Software Development” domain model (in the way it applies to your project, i.e. tailored to that) into subdomains. Then you have “Federated Proxy” in its own bounded context, i.e. as a supporting subdomain and you have “Federated Development” as your core domain.

Now you can have 2 subprojects, CoderGlue and FedeProxy, where the latter has broader, more general applicability and on that basis might also attract more contributors. Which is good, because you want to focus on your core domain, where you add most of the magic and product USP’s. In any case it is good to know that the proxy part is clearly not your core domain, in a similar way that e.g. User Management isn’t.

Note that there are more supporting subdomain candidates like that (that probably don’t need subprojects, btw), such as aforementioned Community domain. With this you can model all the different stakeholders that circle around a software development project (interacting in Roles, Governance and Processes, as well as Socially… more subdomains in your Context Map breakdown possibly).

When done well, a breakdown like this will lead to modularity and extensibility in your codebase, and a clean MVP from where your Roadmap takes flight.

Note that with my as-yet-unannounced projects Groundwork and Outreach I am doing exactly the same:

  • I know that my top-level domain eventually will be very elaborate with many subdomains.
  • Based on that I’ve chosen Go-Fed as it allows me to model these subdomains in separate vocabs + codegen.
  • My app foundation Groundwork has the requirement for subdomains to be added gradually over time.
    • Modularity and extensiblity are key requirements.
    • Dynamic pluggability is a nice-to-have initially, becomes requirement in future when there’s a large installed base.
    • Polyglot service modules is a nice-to-have, but breaks open the ecosystem for anyone to add value.
  • So Groundwork has Service Management as core domain.
    • It has Federation, Users, Vocabularies as subdomains among others.
    • It has merit to be its own independent subproject, because there’s applicability for any application type to be built on top.
  • Outreach is modeled on top of Groundwork as a service module.
    • It constitutes a MVP and offers “Grassroots Campaign Management”.
    • As such it models Community and Campaigns domains on top of Social (AS/AP) domain.

Both Groundwork and Outreach are interesting for different people. They can be independent projects. For me they are just the start towards a more elaborate federated platform.

Going wild: Visualize the software development lifecycle

This is just me having another interesting showerthought to brainstorm on… @cjs of #software:go-fed tooted the following:

So I’ve been wondering what kind of app to build on top of go-fed for a long while. […]

To which I responded, quoting:

How about a foss and federated https://miro.com / https://mural.co where people collobarate on an unbounded canvas.

Sticky notes, diagram / model types are persisted as OWL-defined vocabs.

One of many applications is a go-fed vocab designer that has process baked in, e.g. starting with an Event Storm, Domain Modeling, ending with codegen, persist in a code forge.

This is another example where thinking in (sub)domains as building blocks might yield a whole bunch of exciting things to emerge, and this can happen in parallel by different project + teams.

  • An instance of this tailored to Go-Fed would be a tool to develop for Go-Fed (in an interesting dogfooding manner).
  • The application itself has universal applicability in numerous collaborative ideation and modeling scenario’s.
    • It would be a great developer tool to offer with Groundwork.
    • It would be a unique way to represent the live software development lifecycle in CoderGlue.

Thinking way future here, but how cool would it be to offer a Prezi style overview + navigation over your entire lifecycle and drilldown wherever attention is needed. Instead of static presentation elements you’d see live widgets to interact with both the project itself as well as with the community and those around it?

Note there are at least 3 projects that I know of that approach the UI-side of the equation in this design. They are nodenogg.in, Wireflow, and AutoMerge Pushpin.

My initial (and bad) idea was that Pierre-Louis and I would work together on the fedeproxy project but on different forges. That is Pierre-Louis would work on the https://fedeproxy.eu forge and I would work on the https://coderglue.org forge. And Pierre-Louis would not have an account on https://coderglue.org and I would not have an account on https://fedeproxy.eu. With these rules in place, the only way for us to work together would be via forge federation… which does not exist yet. We would have to manually do it all and document what we do. For instance, I would need to send a mail to Pierre-Louis asking him to create an issue on my behalfe, on his forge and vice versa.

The idea is good in the sense that it demonstrates by example how difficult the life of a Free Software developer can be when there is no federation between forges and that something (anything really) prevents them from opening an account on the forge in which a Free Software project lives. The problems are that:

  • It is very high friction and would effectively slow down any progress we would make on fedeproxy to a halt
  • It duplicates the existing showcase about what it means to work on the Ceph project without having a GitHub account that is documented on a regular basis (see episode 0 and 1).

This initial idea was abandoned and another emerged. A fictitious Free Software developer would be created and work exclusively on https://coderglue.org:

  • They would never create an account on https://fedeproxy.eu (the reasons why they would not do that are irrelevant)
  • They would have an infinite patience
  • They would contribute to the fedeproxy project by reporting bugs and proposing patching
  • They would exclusively work and communicate with the outside world via the tools (forge etc.) that are setup on https://coderglue.org

In the beginning the life of this fictitious Free Software developer would be immensely miserable because of the lack of forge federation, but they have an infinite patience and they would not mind :stuck_out_tongue: Over time, as fedeproxy progresses, their life would become easier. The benefit of this semi-artificial setup are:

  • It is an self contained production environment for fedeproxy for manual experiments
  • All meaningful actions undertaken by this fictitious Free Software developer can be transformed into an end-to-end test and included in the fedeproxy CI
  • It can eventually (end of the year) become a production environment for real users. There would be two publicly available services from the start: fedeproxy.eu & coderglue.org. In addition to people being able to conveniently and easily spawn their own self-hosted service, of course.

While I think I now understand (thanks to your patience and explanations) the benefits of thinking in terms of domains, my brain currently works differently. A colleague of mine also likes thinking in terms of domains and is very skilled in software architecture rooted in domain driven development. It percolates and changes the way I do things, gradually. It will however take time and I suspect other people contributing to fedeproxy have (and will have) a very diverse range of development/thought process.

One of the benefits of a user centric approach and my motivation for starting fedeproxy with a user research phase is to reconcile this diversity and help people focus on what matters to the user first. As an example, let say that the user research report provides evidence that (for instance) what matters most to users when it comes to working with more than one forge is that individual issues and merge/pull requests are federated (and not necessarily the entire software project). Making that happen can be done in many different ways and many different thought process: they need not exclude one another as long as they contribute to what the user actually needs.

In other words, I hope that my creation process and yours can co-exist and that you’ll not be too disappointed by how I’ll do things going forward :sweat_smile:

My feedback on this is two-fold:

  • You started with a very restricted use case: People unwilling to interact with another forge for ethical reasons.
  • In the new setup you are imagining a fictitious scenario. It is not real, or practice, but will influence your thinking.

This while working across forges in different repo’s is a commonly occurring thing in software development. So why not have e.g. a Gitlab and a Gitea instance, and see how your process could be as-smooth-as-possible without FedeProxy, and then look deeper into the added value that FedeProxy could provide to that?

You are then process-focused and the use case where ethical objections disallow interaction with a forge come into the picture anyway as you go along your roadmap.

Yes, it takes time, as it did with me. Also in learning about DDD there’s a lot of formal extensions to the approach, but they are entirely optional. The high-level concepts are what’s important, and they are incredibly intuitive and easy-to-grasp after a while.

Also important is that, while a strategic domain analysis is very valuable at the start of a project, by no means it needs to be applied throughout the further development stages.

The decision can be made on sub-domain by sub-domain basis, and the strategic analysis is the indicator of what the best approach is on each of them. For example a User Management subdomain, if it has just basic features, might be best implemented in a simple CRUD design (or you may just integrate an off-the-shelf library for it). The Federation component might be some complex non-DDD business logic. But modeling e.g. the concept of Project Templates on the other hand, might be better suited for DDD all-the-way. Etcetera.

These people are domain experts, and their processes are to be captured into your domain as well   :wink:

I’m sorry for presenting this out of order. This idea of two forges and this fictitious user only makes sense if it is grounded in the results of the User Research. It would indeed be wrong to base this on my own use case because, as you mention and as is demonstrated by the research (so far), turns out to be extremely rare: people generally do not refuse to have a GitHub account. However I think it is very likely that the User Research result will show that Free Software developer routinely need interactions with software projects that reside on forges on which they do not have an account, most often for practical reasons (too much friction mostly). If (and only if) it turns out to be what emerges from the User Research, it will have the same practical consequences as my particular use case. Otherwise… well, the fictitious user idea will be discarded altogether :slight_smile: In any case it will not be implemented or started before the User Research is over: that would be backwards.

I’m still struggling to understand how to work with that. I don’t have a clear idea of what the “Software Development” domain means or even the “Free Software Development” domain. I do understand things that relate to both, because I’ve been in the field for decades, but I would have no idea how to split them into sub-domains. While I get the general idea of Project Templates, it would be very difficult for me to explain it to someone else. And even more to figure out how it can usefully be applied to the “Free Software Development” domain and to the forge federation. I’m enumerating my confusion in the hope that, since you have clarity, you can tell me what I could do to get better.

1 Like

Like I said before, you are on very good path in terms of gathering feedback. The ‘domain expert’ part was said with a wink because you are getting there one way or another. Words and sentences from these interviews will be transformed into types and code, and by that you’ll have defined your project’s ‘ubiquitous language’. DDD is only there to help ensure a clean translation and a continued correspondence to things your stakeholders understand.

Ultimately in most cases the domain model is that part of the abstraction of the real-world expertise and practice that is relevant for the application. I.e. it is tailored for what you want to offer, containing only those entities and relationships where product functionality will be delivered.

But for fediverse in general, and FedeProxy in particular, you have some additional non-functional requirements, that demand extra care in the domain design. Interoperability requires a model that is broadly understood and accepted, that can be readily reused. Modularity and extensibility are other NFR’s that make this very important. Some entities will directly translate to AP message formats. Note that both AS/AP and ForgeFed - without using DDD terminology - did just that. The domain is directly visible in the vocab data models here. Other entities + relationships may be directly tied to UI widgets to manage them, or be reflected in an API design that allows other projects to provide extensions.

How you split your domain is up to you. The decision to split comes naturally during elaboraration. In an ECommerce domain it makes sense to split between Ordering and Shipping and Billing. They are different processes and even different departments are often involved.

If a project was building e.g. a Continuous Integration product, then a code Commit might trigger the CI Pipeline to be set in motion. CI is the core domain, and Commit may be part of it, but it might just as well be Trigger, or whatever. From the perspective of the CI project a relationship with Code Forges may be designed by means of a supporting subdomain. It is only supportive, because not core to what may trigger the pipeline, and there can be multiple different triggers.

When this project in addition to CI also offered features related to Code Forges domain, then Commit would be defined in both bounded contexts, and may have different properties. Only those that make sense within the subdomain. But there is a relationship and one can be correlated / translated to the other (in your DB they may be persisted as one and the same entity).

Some examples in ForgeFed about the Context Map and bounded contexts. Their core domain is Source Control Management, and not Issue Tracking. They define a Ticket (their ubiquitous language choice, where I am used to ‘Issue’) within the core domain, but not a Comment on an issue, or a Milestone or Release.

ForgeFed defines a new Actor type Repository as an independent entity to interact with (i.e. an agent), but does not include Person actors or Organizations, which are already part of the Social Networking supporting subdomain. In a project based on ForgeFed a Comment will maybe also be mapped to this domain, i.e. by representing it as a as:Note object (whereby assuring that e.g. Pleroma / Mastodon may understand it, if it were sent as a toot).

In FedeProxy you’ll make similar decisions. If you want to highlight the people involved in software development and their interaction, then you’ll build on top of Social as well. In this domain you have e.g. Person and Group actors. Quite bare in their definition. Question is: Is that enough to model a project’s Software Development Community? I’d say No. People have well-defined Roles, and as they change hats they are involved in different activities.

Based on this you may reuse the Community vocabulary that is being discussed elsewhere in this forum. You may give the entities different names / labels. Is Project part of the FedeProxy domain? And does a project have Members? Is the membership the same as the Community membership, or are they different? Is Community an intricate part of the FedeProxy domain? Can membership be divided into Teams? Is there a Development Method in the domain? Are team Roles dependent on the Method?

All these questions can only be answered by gradual analysis and elaboration. You only take what you need, keep it high-level and fill in details and blanks as you go along. Things all start by finding the ubiquitous language, keep lists of terminology and write simple English sentences to describe relationships.

From this you scrap and refine, until you have language that represents a MVP and possibly a Roadmap from there onwards. It will naturally translate into code abstractions, and also provide many handholds as to the architecture that is needed to support it.