Writing documentation up front is a good idea, but only if one treats it as the first iteration of the program with the expectation that the initial documentation may well be thrown out. The process of writing the documentation no doubt will help you organize the plan for writing the code. But there will also be things that the documentation does not capture or problems it does not foresee that emerge as the program is being written.
The danger is that either you fail to update the documentation to account for the changes in the system that emerge in development or you only update parts of the documentation as you go which causes the documentation to become inconsistent and unreliable.
A middle ground is write the documentation up front and then rewrite it after the system is done. The initial draft helps guide the design and the final version captures the full and complete essence of the finished program, which is nearly impossible to do up front.
> Writing documentation up front is a good idea, but only if one treats it as the first iteration of the program with the expectation that the initial documentation may well be thrown out. The process of writing the documentation no doubt will help you organize the plan for writing the code. But there will also be things that the documentation does not capture or problems it does not foresee that emerge as the program is being written.
Everyone else is building all kinds of things from technical drawings. We could identify and describe what is so different about writing code which would make it clear what not to document or we can just suck it up, stop pretending our problems are special and closely mimic existing methods :)
Making a construction drawing is hard but people are some how doing it, they make them for mind blowing size projects. Things definitely go wrong but no one says it is just the way things are. If changes are made there shall be new drawings.
However, when it comes to renovation or repairs the process is often the opposite. Unless it is a boat, there if every brush of paint is meticulously documented it adds value in $$$.
A friend told a funny story about construction workers building a large building across the street while they were debugging their code. First he joked about construction workers doing real work, then about how structured everything was and eventually about the building being finished while they still had countless bugs.
> Things definitely go wrong but no one says it is just the way things are. If changes are made there shall be new drawings.
> then about how structured everything was and eventually about the building being finished while they still had countless bugs
There is a famous story of the Hyatt Walkway Collapse where a change was made, was improperly documented, and caused a production bug that led to many deaths. Extreme example but production bugs in physical construction are extremely common.
Construction has a much more mature building process, much of that is probably related to the cost of replacing things. We even get to cut and paste :)
That last document seems like something that could be done in software development. Their list (starting with the worse) is dominated by mistakes pouring concrete.
- Inadequate planning (from task through to project level)
- Late design changes
- Poorly communicated design information
- Poor culture in relation to quality
- Poorly coordinated and incorrect design information
- Inadequate attention paid in the design to construction
- Excessive commercial (financial and time) pressures
- Poor interface management and design
- Ineffective communication between team members
- Inadequate supervisory skills
For software we would have to gather the data similarly. Then you put it all into an LLM along with the employees previous assignments and code, pull the lever and the answers come out.
Ah, yes. The aim should be to have pure information so that insurance companies can pick up the bill. In order to accomplish that the system should prevent the employer from meddling in the process. Working from home seems required to properly benchmark the employees.
This lines up perfectly with my use of the documentation. It really helps quite a lot to do it first, but as the implementation is underway, new scenarios come up and it's often a challenge to go back and update the docs. The new things also aren't something that derails the entire design. In the end, the docs are usually thrown away or transmuted into something useful for other departments.
>but only if one treats it as the first iteration of the program with the expectation that the initial documentation may well be thrown out.
This point really hints at there being some other facotr at play. I agree that writing documentation first is fine, but in the same way writing some bad code is fine.
I'd argue that the most important thing you can do to increase your chance of success is getting to the details as fast as possible. Do everything you can to get out of nebulous "big idea" mode and get into "we have this, what exactly are we changing about it" mode.
If you write documentation or code doesn't matter.
Another nice benefit of doing it this way is that it allows the two sets of documentation to follow different formats and levels of detail. Which can ultimately be less total effort than trying to engineer a functional multitool.
The questionable reliability of documentation is the big problem, for sure. It would be nice to be able to collect metadata such as referral dates and even comments. If you visit the docs, tick a box that says, "I referred to this and it was accurate," or, "This needs to be updated," etc.
I think that the usefulness of any “X driven development” methodology is so highly variable based on the particular problem being solved and the particular brain solving it, that it’s impossible to make any general recommendation for anything. Like everything else in programming, it depends. Documentation driven development might be great for one person, but for another person it could be a tedious waste of time. Everyone should feel free to find their own way. If it works for you, good. If not, let other people do what works for them.
I think a lot of XYZ Driven Development advice can basically be boiled down to: write down some kind of plan before you start coding.
Certainly one should think about the design before implementing, but I think writing acts to force active reflection on the design - be it a plan, or documentation, or a test.
At this point my approach is to plan out my current ticket into TODOs before I make a branch. I do this in phases until I get down to the level of what functions need to be modified/made. If I know there is a function that is going to be doing a bunch of branching or is otherwise complex, then I know it will need a good bit of unit testing, and I might opt for TDD - though its rare. I'm used enough to planning my tickets now that it is usually sufficient to get to a good design.
I would recommend trying to do some sort of planning of your work. It can be hard and time consuming it first but it can be a superpower if you get into the habit of it.
Planning upfront like this inevitably leads to better design and less rework. I'll see that steps conflict and rewrite the plan before rewriting the code.
We are trying out some DDD for an upcoming pigweed.dev tutorial. I'm docs lead for the site and I'm working with 5 or so engineers. It's going decently. As a technical writer, it's hard to flesh out exact content until the features are built out. Especially in a tutorial, where each section builds off the ones that came before it. So we have a lot of placeholders in the tutorial. All-in-all I think it's been fairly helpful for me to repeatedly say: "This will be our main artifact. This is where people will learn of your features. Please make sure your work and features are captured in this document, otherwise they kinda don't exist (because this doc is where people will most likely learn of the new features)." I don't ask them to fully flesh out docs themselves; I'm just asking them to dump notes into the draft doc and insert commands/code as things become fleshed out. I think it's also a useful focusing exercise for the team - there's so much we could work on but if it's not going into our primary documentation artifact and it's not foundational work that is prerequisite to what we want to show in the tutorial then that sometimes makes it easier to decide that the thing is lower priority. There was also an interesting feedback cycle where the team worked on some really important stuff that is usually kinda "behind-the-scenes" and low-level, and we asked ourselves "we have to work this into the doc and make people aware of it... how do we do that?"
For designing APIs, my favorite approach is a hybrid of the two. I'm not sure if there's an official name, but I've taken to calling them walking skeleton tests.
The gist is that you start by writing example client code demonstrating small but useful interactions with the API, including explanatory comments. They won't work yet, because the API hasn't been implemented. But you're already in a position to talk them over with your team and start thinking about user experience issues. And they give you a nucleus around which you can start building high-quality user documentation.
And they're your automated end-to-end acceptance tests.
Once things look good (not perfect), then you can start implementing the API. For this portion I don't necessarily do TDD, because I remain unconvinced that writing unit tests first is inherently better than writing unit tests afterward. But I might. Depends on my mood. But I do keep running the walking skeleton tests, which serve as my primary feedback mechanism for evaluating implementation progress and success.
The more important thing is that I can start stepping through those walking scenario tests and seeing how well the interaction actually works in practice. And I can use those observations to help inform an ongoing conversation about the API's design and functionality. I can also get quick feedback on whether some aspect of the high-level design isn't going to work as well as we had hoped. Ongoing updates to the design are memorialized and communicated by updating the walking skeleton tests.
> For this portion I don't necessarily do TDD, because I remain unconvinced that writing unit tests first is inherently better than writing unit tests afterward.
I would argue that TDD doesn't imply writing unite tests - it's about writing any tests first.
If you have acceptance tests already written out, you're already doing TDD.
This is actually how software and systems engineering works in life sciences.
There are a set of artifacts which are designated the validation artifacts which includes a user requirements specification, functional design specification, technical design specification, and configuration specification.
Before any release, the team will start by identifying which of these documents need to be updated and which sections need to be updated. This is codified in a formal, signed document.
Then work starts and an updated version of each of the document is created. Prior to releasing the software, the validation process kicks in and basically verifies the correctness of each of those documents in reverse order: the configuration specification -> installation qualification, the functional design specification -> operational qualification, the user requirements specification -> performance qualification (there is no explicit qualification of the technical design). The qualification phase has a mountain of evidence of testing.
The result of this rigor is: 1) the products tend to have very low defect rates because every change is accounted for, 2) the release cycle can be as long as the build cycle, 3) it's very hard to fix bugs "on the fly" as these then need to be documented as a deviation and accounted for, 4) anything that breaks is easily accounted for, 5) it's very, very hard for startups to succeed in this space because of how much rigor is required.
I initially hated it because I wanted to move fast. But after leading a few release cycles, I could see a lot of the benefits as we rarely "built the wrong thing", we rarely had major defects, and work-life balance was generally pretty good because every release cycle was meticulously scoped since we had to start by describing the planned changes starting from the documentation. There is less scope creep once work on the release starts since the start of the release has already identified what will change and adding scope means adding more paperwork after the fact.
The author seems to conflate several different types of documentation, like requirements, code comments, and api explanations. Additionally TDD isn’t intended to work in the absence of any documentation, the concept is that a test models the requirements.
Maybe a paradigm to consider more is:
Literate programming (LP) tools are used to obtain two representations from a source file: one understandable by a compiler or interpreter, the "tangled" code, and another for viewing as formatted documentation, which is said to be "woven" from the literate source.
I honestly believe documentation is more important than tests in some cases. The time spent by a dev every time to understand why a method was created that way adds up quickly. The what is more evident from the code.
The idea of writing documentation before the code is interesting, it's thinking out loud
"During the early phase of software development the documentation is the specification and is the design... If the documentation is bad the design is bad. If the documentation does not yet exist there is as yet no design, only people thinking and talking about the design which is of some value, but not much."
"During the testing phase, with good documentation the manager can concentrate personnel on the mistakes in the program. Without good documentation every mistake, large or small, is analyzed by one [person] who probably made the mistake in the first place because [they are] the only [person] who understands the program area."
"Following initial operations, when system improvements are in order, good documentation permits effective redesign, updating, and retrofitting in the field. If documentation does not exist, generally the entire existing framework of operating software must be junked, even for relatively modest changes."
All quotes from "Managing the Development of Large Software Systems" by Winston Royce, 1970.
All quotes perfectly match my experience working on projects that were developed using contemporary low-documentation agile development methods.
Wholeheartedly agree, however... It's well accepted that most developers would rather chew off their own arms than write documentation. The acceptance, sort of, of TDD shows that developers would rather code even in pseudo code test languages than write narrative prose.
I write a "functional description" that describes the functionality of each screen from a user's perspective. It also contains technical details to jumpstart devs but the main focus is on describing how the software works.
Oddly enough, I write these in Word and keep them on SharePoint. This allows the team to add comments, maintaining the historic discussion around certain functionality.
I really love this idea in theory, and I believe that for some system, specially mature ones, it may work well. I see good documentation as a super power; it empowers readers and motivate people to understand more about the system without being caught in the weeds of reading the source.
The source has baggages, and the intent of every single function calls is not always evident. Writing documentation up-front can help direct the source, but this is a tug-of-war environment. Each affect the other in its own ways.
And for that reason, documentation driven development can be a real drag. You start writing documentation with the best intentions, everything works great for this first release. But 2 months down the road you need to modify something and it has a ripple effect on many of the things you documented. It's a non-negligible cost.
I've been working on this open-source tool(https://github.com/pier-oliviert/sequencer) and I've spent a lot of time on the documentation. And what I described above happened. I wanted to make a not-too-big change, and it required me to rewrite 30% of the documentation. I still love the documentation aspect of it, but it definitively has a higher cost than tests, in my experience.
Would be interesting to see what impact some sort of llm system might be able to have on this approach. Making some changes in one area of the documentation, then having a system indicate some other likely areas of impact which you might not have considered, seems like it would be useful.
If the model can reflect the existing documentation that well, w.r.t. changes in code, you may as well use the LLM as the documentation by fine tuning it with boilerplate/initial docs and have it generate the future documentation based on a context of just the code.
My attempts at this (DDD) involve choosing system components or protocol/API layer dependencies such that they can generate the documentation when they generate the client-lib and server interface code. Isn't always possible, and doesn't cover everything. Having a comment-doc pipeline also helps but sometimes that just adds more trees to occlude the forest. This way the documentation doesn't lead the code too much (I'd rather see deprecated warnings than Not Yet Implemented errors).
Good team communication and a solid codelab for new members can go a really long way, too.
> it has a ripple effect on many of the things you documented
Welcome to the world of technical writers :D
(In other words, I'm suggesting that this is always the case and it's not something peculiar to DDD. I would argue that DDD has just made you more sensitive to what we go through every day. But yes, if you try to create a final doc and the features aren't stable, you're going to have some wasteful churn.)
Rational Rose UML
RequistePro.
Doors.
Just plain prototype something through many iterations until is good then document.
Doxygen
All of the above except the prototyping a waste of resources more or less.
If I had my software shop (which I don't) I would expect a software module
to come with a README.md Which a brief explanation of what is is
and no more than 3 step instruction
for a reader to see the software module running and doing something useful.
Given that the article is focusing on designing and developing APIs, I'd argue that, done correctly, documentation is a prototype. It's functionally equivalent to paper prototypes and low-fi mockups in UI design.
The thing that makes these kinds of artifacts so nice compared to just diving straight into cutting code (specifically implementation code - see my comment elsewhere on walking skeletons) is that they're cheaper to build and easier to throw away. Which lets you iterate more rapidly in the earlier stages of the project, with less risk of accidentally anchoring yourself to decisions that get made before you have sufficient information to support a firm decision.
That's also arguably where the various design and documentation tools and techniques you listed fail. I haven't used all of them in anger, but at least by appearances they're all relatively heavyweight and expensive things that are generally ill-suited to lightweight early planning in anything short of a situation where you're dealing with a CMMI requirement.
I agree the API that is the API is done correctly, it provides honest guidance.
Once upon a time when 3G mobility was all the rave I was with a company named Lucent which was vying for a contract with Japanese NTT Docomo. The job: Provide BTS (Base Transceiver Stations) for their brand new network. Hardware and software.
Their API documentation was an Excel Spreadsheet with c function call names, around 300. Column 1, the function name, column 2 the expected parameters in byt form, the return values in byte form and last column..the maximum time limit to return the call in milliseconds. The possible error codes (lots and lots of error codes) The API was also shared with Hitachi which was the competition.
The API read like a Tax Form but the expectations where so well understood.
Down to the bare metal.
> Just plain prototype something through many iterations until is good then document
This is always the best. Iterative prototyping gives you continuous hindsight over design and implementation, _nothing_ beats that.
Unless you're doing something trivial in which hindsight would only serve as temptation for yak-shaving. Then it's not so good. Unless you don't have deadlines, then it's good again.
Many of these arguements are moot – because for any top-down function definition technique, the How to Design Programs recipe covers virtually all the bases.
Between a signature, purpose statement and examples, you've declared most of what documentation provides short of a longer contextual statement of the functions role in a codebase.
I wouldn't focus on the fact that the examples are on single functions. Libraries and frameworks are about API design and not individual atoms, and writing your documentation ahead of time is about crystalizing the API vision you have in your head.
One may have accidentally glossed over a critical design decision and this process can surface that.
Few can afford not to follow the basic (DO --> SECURE --> DOCUMENT --> ANNOUNCE) workflow. Working in any other order and you end up having to redo some if not most of the previous steps for whatever you where enterprising.
So why can't I use documentation as a requirements spec and just feed it to a LLM for implementation ? With lots of examples in the documentation that can serve as test cases for the generated code.
The problem is fundamental: you need a certain level of mind meld to be able to get away with leaving things unstated and assuming the other person will know what you mean, or can figure out the right thing to do for themself. When you don't have that, you have to compensate by making your explanation of what you want as specific and precise as possible. Which, just like in the story The Monkey's Paw, turns out to be infeasible because of the fundamental sloppiness of natural language as a communication medium.
The level of mind meld you need to make natural language communication behave well comes out of domain expertise and a certain amount of time spent working together and getting to know each other. Outsource developers and LLMs both lack it for reasons that are admittedly different, but comparably unfixable.
that's still to be proven. At the moment it can't do what you describe for any meaningful piece of functionality, there's no certainty that it will be able to anytime soon
If you're willing to write a very detailed requirements spec you can just... write ruby or python code in a framework and have a working application...
Except for trivial examples, all real development needs to be undertaken iteratively. Design a bit, build a bit, rework both based on what you found out. Unfortunately most people - and definitely most organisations - don't want to rework anything as it sounds like a waste of time/money rather than an essential step to take working code and turn it into production code. So instead they look for silver bullets. "If you just do the steps in this order..."
Everything is already natural language driven development, but management/execs often like to avoid accountability by prompting their human LLMs verbally off record.
That's mostly how Knuth codes (with a flavor twist in the form of literate programming). It's a huge endorsement of the idea presented in the article :)
> Making sure tests pass can be challenging to handle with the unknowns of implementation detail. After all, if you expect parseInt to act one way and it behaves another, you will likely have to rewrite all tests that worked off that assumption.
Isn't this kind of the point of TDD? TDD makes it harder to write tests based on assumptions about implementation details, making it less likely that you'll make such assumptions. You end up with less unnecessary coupling between your tests and the implementation details of the code they test, which means fewer false failures (tests that fail because the structure changes, rather than the behavior), and a more robust test suite.
The danger is that either you fail to update the documentation to account for the changes in the system that emerge in development or you only update parts of the documentation as you go which causes the documentation to become inconsistent and unreliable.
A middle ground is write the documentation up front and then rewrite it after the system is done. The initial draft helps guide the design and the final version captures the full and complete essence of the finished program, which is nearly impossible to do up front.