Software development and the waterfall model

As mentioned, I recently read The Design of Design by Fred Brooks.

He covers a lot of ground, albeit not in any rigid fashion. As such, it is not a handbook, but more a collection of thoughts about the design process, most grounded in personal experience. Fred Brooks knows his stuff.

An interesting point he makes is that the waterfall model of software development is completely flawed and should be abandoned.

In short, the principle in the waterfall model is reason - good solid thinking: first we figure out all what's needed in a given system, then we figure out all aspects of how to build it, then we build it.

He has several arguments for why the model is flawed. One of the simple arguments is that it doesn't model how good designers work. Naturally, if you want everyone to follow a certain mode of operation, it would seem like a good idea to copy the mode followed by the good ones.

For me, the biggest flaw of the waterfall model and its companion, the big fixed-price project, is that it requires one to make the decisions up front where the uncertainty is greatest and the consequences of each decision most remote.

This shows up even in the earliest phases, e.g. when discussing whether to include a certain requirement or not. Inclusion may mean that the project will run over budget and schedule and produce a completely impractical never-used feature. However, the budget overrun may not even be visible in the design phase, but only show up late in development, and the futility of the feature may not show up until years after deployment where it is discovered accidentally that nobody uses it.

An important point in the book is that the design process is a learning experience, a process where one learns what's needed and what's possible. And what's elegant and what's good in the particular circumstances. I'd add that the same is true of the embodiment of the design, the code itself. Often people won't understand a system properly until they sit in front of it and have to start doing what they need to do. Deployment is always a learning experience. And similarly, it can be hard as a developer to assess the consequences of a paper design. When the code is written, maybe that fancy idea everybody liked beforehand isn't that hot afterall.

If one doesn't allow for learning, both for the receivers of the system and the people who deliver it, it's really difficult to get the software right.

By this thinking, the rational model becomes a bureaucratic way of delivering what might just turn out to be the wrong system.