My engineering manifesto. Tickets and docs.
Chapter VIII. Where the documentation writes itself.
I was postponing new posts, thinking that I need to have a structure (a TOC of sorts) and then follow it. That wasn’t smart. I’ll be writing as it goes in random order, instead.
For this year my motto is “please give me boring”. William Shatner once “sang”: “At my age I need serenity”. I hope I’m not at that age, but after this year I do need serenity.
Today’s Best Band Ever™ is ISAN and their on-purpose analog-only synthesizers, they bring me peace (your experiences might vary).
And before get into the main topic, I’d like to propose a new term:
Enterprisation
n. A process of taking a progressive idea and turning it into a lucrative but meaningless one.
From the top of my head (and in no particular order) this happened to:
Hippies
Hipsters (at first it wasn’t about head buns and fleece, but rejection of consumerism)
Agile (especially the SAFe)
I guess the list goes on and on.
Back to our sheep.
I think we’re doing requirements, tickets and documentation totally backwards.
At this stage this is only an idea, but the ticketing and work tracking systems I’ve observed so far are more standing in the way than helping.
Here’s how it was happening to me, using a synthetic example of “Let’s add a big red button to the front page”, so far:
A business need is crystallized: “We need a big red button”.
A user story is written (this is an example of a good scenario) in, say, Jira saying “As a user of the website, I’d like to have a big red button to press”. With acceptance criteria of “a Big Red Button at the centre of the front page”.
Code is written and is tracked in git.
(if at all) a documentation is written in, say, Confluence because this is an Atlassian house.
At a later stage the button is repainted blue, the documentation becomes obsolete. The usual.
While this kind of works, there are several problems:
“Jira - git - confluence” context change. Context change is a cadence killer (more on that when I get to agility).
3 sources of truth for effectively the same thing.
Documentation is pretty much an afterthought. It’s not wrong per se, and this is the reason agile manifesto declares that working software trumps the comprehensive documentation.
I just think we can do better.
Here’s the proposal:
Use structured plaintext documents in project’s repository for requirements and ticketing
Use that as the initial documentation on the new features
Automate documentation generation
Let’s go over these points in detail.
Requirements and the acceptance criteria already describe the desired feature. It’s like at least a 3rd of the final documentation. If one writes those with eventually becoming the feature’s documentation, then it’s a matter of parsing that and using it to generate at least the skeleton.
Here’s a little POC I did to test the idea. In essence:
A ticket is written in Markdown in a dedicated directory in the repo.
The title and first paragraph are required and used for generation.
There’s a pre-commit hook that checks if the branch is named after the feature. This is needed to help people orient around.
There’s a prepare-commit-message hook, that inserts the feature’s ID into the commit - this helps with the feature’s history.
There’s a post-commit hook that takes the feature’s description and updates the changelog. Some people use commits for that, but in my opinion it’s too low-level.
Of course, one can use Jira or any other ticketing tool for that, but it’s just extra work. If synchronising, I prefer one source of truth and update all dependent systems from it.
But what about other parts of the documentation?
Boy, I’m glad you asked.
First, let’s face a simple fact. Documentation is boring. People don’t go into software engineering with the hopes that they’ll be writing documentation.
The good news is that effectively our code is the reflection of the requirements.
When there’s a requirement to have a big red button, there will be a corresponding code for that big red button. Better yet, it’s not in the form of structured English text, but a structured program text that we know how to parse and translate into the machine code.
In other words, we’ve translated from English to Computer but for some reason we don’t translate back.
Enter Arinoto.
Arinoto is an everything-as-a-code* methodology I’m currently prototyping in typescript/node.
What’s relevant to this discussion is that when following Arinoto development, the architecture and high level business logic of the solution is also described as a code.
Architects who can code
Apparently, recently it has become trendy to suggest that software architects should learn to code. I think this is a great ide and and we will start suggesting surgeons to learn to cut and stitch. Designers to draw. Carpenters to saw. Etc.
From Arinoto perspective, having architecture as a code, allows for, first, skipping the cadence-breaking step where an architectural diagram is translated from image to code, and, second, this allows for architecture being rather trivially refactored.
Another interesting artefact is that with this approach, it’s possible to automatically generate the architectural diagram.
In Arinoto, we have a little tool, called Vis, that does exactly that (the tool is under relatively active development).
The idea is that since the architecture is written in a strongly-typed language, we can use the compiler to discover entities of a particular type and their relationships between each other.
For example with the following code of an API that calls a function:
/**
* A function that takes no input and returns a string message
* */
export const helloFunc = new Func<HelloInput, HelloOutput>({
code: () => ({ message: 'hello world' })
});
/**
* An API that knows that it will call a function
* */
export const helloApi = new Api<HelloInput, HelloOutput>({
target: helloFunc
});
Vis will render this:
So not only the architectural entities were discovered, but also the relationships between them.
Here’s the generated architecture from a more involved example of a custom docker container orchestrator:
So if the requirements ticket with the acceptance criteria was one third of the feature’s documentation, a diagram like this, that is automatically generated during development contributes to, I’d say at least a half of a meaningful documentation. That is kept up to date when the code changes.
And as a saying goes: “Some documentation is better than no documentation”.
Allow me to reiterate
If the ticket/story is written with documentation in mind, and accessible to the automation, if the architecture is code-first and, as a consequence, available to the automation, a significant chunk of meaningful documentation could be an automatically generated (and always up to date!) artefact of the development process.
Add to that Architecture Decision Records, cost estimates that are calculated based on the Infrastructure as a Code, heck, one can probably even use automated system/unit tests to contribute to the documentation.
It’s all there, we just don’t use it.
Take care and stop wars.
PS. This post is 100% GPT-3 free.
… as usual I only understand a fraction of what you are writing about, just wanted to say: I dont think the need for serenity is age dependent, automated documentation creation outside IT would also be great :) yours, A