Building Better Software

A reflexion on how we can build better software by analyzing the problems, rethinking the whole process, and proposing a solution.

The problem

The way most teams and companies build software is far from ideal. We have an almost unlimited set of tools, languages and power, and yet, the software we use daily has never seemed so broken and slow. So what is wrong ?

The main problems reside precisely in the tooling but also the methodology. We build software like an amateur carpenter builds a cabin in their backyard : no design, no plan and pieces glued together in an inconsistent way.

Software Engineering is one of the only engineering fields (compared to car, airline, construction...) that feels so "Amateur" (exception for Boeing though). There are lots of reasons for that, including bootcamps producing amateur developers, poor project management methods like Scrum, projects led by non technical management... and so on.

The consequence of this amateurism is that the way we build software is too "concrete" (cf. Conway's Law). It means that the code we write is too entangled and close to the end result. Therefore, there is absolutely no reason to keep doing the following things :

• Having repositories representing the underlying technology (a repo for Spring Boot, a repo for React, a repo for iOS, etc.)
• Having repositories representing the "thing" that will be deployed (API, Website, Mobile app, etc.)
• Thinking technology first and not about the "What/How"
• Thinking UI/UX first and not about the "What/How"
• Thinking microservices, k8s, etc.

All these biases are mainly related to the fact that a few people are able to reason about abstract things. They need to see something to have the feeling that they work (or not) as expected. And we must be honest, software engineers like playing around with tools and new hype technologies.

A proposal

We need to restore Excellence and Rigor into Software Engineering, and for that, we need more abstraction and new processes.

If we had to define a good codebase, we could list the following properties :

• Upstream designed
• Auto testable
• Auto documentable
• Projectable
• Tech agnostic

Take a minute to think about your codebase. How does it fit into these properties ? How are you feeling ?

To illustrate these, let's take a car engine as an example. Do you think they start building it from day 0, put it in a car and run the car to test it ? No.

First, a list of specs is defined (volume, power, etc.). Then, engineers study it, build a prototype in 3D, run some simulations. Once approved, they manufacture an actual prototype that they test on a dedicated bench by plugging mock gas and exhaust. And so on...

Upstream designed

Following the car engine example, we absolutely need to restore a design phase. We are not talking about writing hundreds of pages of specs in a waterfall cycle do not worry. But more about defining interfaces and contracts and analyzing impacts.

Most importantly, these specs must be instrumentable. Ideally we would use UML but this norm is too complicated and completely lost the fight. Therefore, we need a lighter format (more details below).

Auto testable

Writing tests by hand is an absolute heresy. No wonder why developers hate writing them. And most of the time, they don't even test actual code, but only the underlying technology.

Plus, things got even worse with the advent of ChatGPT and Copilot. Pull Requests now contain thousands of lines of auto-generated tests that are absolutely not maintainable nor readable.

BDD (Behavior Driven Development) tests were a good idea initially but the assumptions they are built on are completely wrong : the business does not know, nor doesn't want to read/write Gherkin. Plus, testing by manual examples defined by the developer makes it easy to forget cases.

That's why, a codebase must be auto testable via simple heuristics defined deterministically from the specs defined above.

Auto documentable

If you take your codebase, are you able to make an exhaustive list of features it provides ? Probably not. Your initial spec documents and documentation are probably completely outdated. Chances are big that no one in the team is aware about this feature hidden at the bottom of this screen, hidden itself at the bottom of the footer.

It goes without saying that documentation outside the codebase (e.g. in Confluence, Notion, etc.) is an even more complete non-sense. It's better to have no documentation than a outdated one.

That's why, like tests, documentation should be made based on the specs and not written manually in a side Markdown document.

Projectable

"Auto testable", "Auto documentable"... are just examples of "Projectable". Behind this more generic word, resides the idea that the codebase must be instrumentable to extract whatever is needed, without any human writing.

As an example, take the GDPR that we all had to implement in 2018. We had to make lists of PII (Personal Identifiable Information) that were managed by applications. This is an example of projection that must be done automatically. Not by browsing the database schema manually and filling an Excel Spreadsheet for the Legal Team.

As another example, let's take a form. It does not make any sense at all to have so many forms in our codebases. Especially when they all follow the same layout defined in the Design System. These must be projected from the use cases.

Lots of technologies make it easy to reason about state compared to behavior. If you take a regular MVC framework, you store state in the database and not the flow that leads to that state. This makes you loose important information. Here again, state should be projected from events. Nothing new here though, as there is already a lot of litterature on ES (Event Sourcing) and CQRS (Command and Query Responsibility Segregation).

OpenAPI spec, Admin backoffice, CLI... are other great projection examples.

Tech agnostic

As our managers, our customers and other people, no one cares if we are using Spring Boot, Rust, Go, Next.js, Remix, Ruby on Rails, Django, PHP, Symfony, Laravel, React, Vue, Svelte, jQuery...

These are just tools and should not be the central piece of our applications anymore. This has to stop.

A solution

Based on these observations, we have been building such a solution for more than two years now. RebootX is based on it for example.

Layers

The idea is to always start by defining the use cases. A use case is as simple as an Input, an Output and the main that runs on the client and/or on the server. With hexagonal architecture and dependency injection, it remains completely independent and infrastructure/tech agnostic.

With only this, we can write our application code, our business logic and test it automatically with the built-in auto tester (remember the car engine tested on the bench ?).

Depending on the size of the team, the work on the targets can be done in parallel. Targets must define how to behave with a use case.

Typically : how to render a use case with input ?

Let's say we have a Next.js webapp, a React Native mobile app and a CLI app (e.g. built with Commander). For Web and RN, we would render a form. If we abstract correctly our components, we can even reuse the same form ! For CLI, we would show a prompt for each input field.

But overall, this does not impact our use cases. They do not care where they are rendered. Creating a post is only a matter of saving content into a data store after all.

The main goal is to do plumbing only once. And then, our use cases integrate seamlessly within our targets.

Conclusion

Software development often suffers from poor tooling and methods, resulting in slow, broken software. Problems include amateur developers, bad project management, and tech-first thinking. To improve, focus on upstream design, auto testing, documentation, projection, and tech-agnostic solutions. Prioritize use cases over technology and streamline processes with proper abstraction and new methodologies.

Interested or just curious ? Reach out at hi@c100k.eu. More details to come.