A magazine about programmers, code, and society. Written by humans since 2018.

Goodness Gracious, Great Balls Of Mud!

On June 25th, 1999, Brian Foote published a seminal article that became, for a short while, one of the most commented pages of the early Internet, at least by struggling software developers and prospective architects. Such was the impact that it “was twice featured in Slashdot” (kids: that is the 1999 equivalent of “this article hit the homepage of Hacker News”). After almost exactly 25 years since that publication, one question still lingers: when are we going to accept defeat?

Nearly all the commentary around the “Big Balls of Mud” article (and there is a lot of it online) orbits around the concept of “Anti-Pattern”. The (purposely shallow) narrative goes like this: Big Balls of Mud are bad, they must be avoided at all costs, and here is a series of steps you can follow to avoid that situation. Yet, evidence shows that the Big Ball of Mud stubbornly returns, time and again. It is winning, and we are not.

And it is not like we have not tried to avoid it. In the 25 years that have elapsed since the publication of Brian Foote’s article (and even before that), many have tried countless attempts at coming up with solutions for the problem: Structured Programming, TDD and Extreme Programming, MVC versus MVVM versus MVP versus MVI versus VIPER, Three-tier, DevOps, CI/CD, SOA, Object-Oriented Programming, Microservices, Enterprise Service Bus, ISO 42020, Yourdon/DeMarco data-flow diagrams, UML, manifestos, C4, IEEE 1471, Design Patterns, Infrastructure as Code, Working with Legacy Code, Martin Fowler, TOGAF, philosophy, Christopher Alexander, reference architectures, quality processes… and the list could continue on and on and on.

The issue at stake here is not, in my humble opinion, the lack of yet another methodology or programming language or agile framework that can deliver us from the dreaded monster. I also do not believe that Brian Foote said it could be done at all.

Quite the opposite, actually: he wanted us to accept the fact that Big Balls of Mud are, just like Thanos or Mr. Smith, inevitable.

So here is what we should all be doing instead: embracing the Big Ball of Mud. Let us be honest; in an age where mental health concerns are finally gone mainstream, I find accepting such truth after 25 years a deliverance for the software engineer in me. Our projects are like hourglasses glued to a table, so we should just breathe.

Brian Foote left many hints of this perspective in his piece; let us see a few instances where he lets this message slip through the cracks:

Even systems with well-defined architectures are prone to structural erosion.

No matter how hard you try, Big Balls of Mud will happen. Behold, the only way to keep it at bay is to continuously work on the software, updating its dependencies, adding and removing tests as needed, moving stuff around continuously:

A sustained commitment to refactoring can keep a system from subsiding into a BIG BALL OF MUD.

This is particularly true during the early phases of your software; just embrace the chaos and move forward:

A certain amount of controlled chaos is natural during construction, and can be tolerated, as long as you clean up after yourself eventually.

The impossible dialogue between software developers and their management is one of the leading causes for the awakening of the Big Ball of Mud:

One reason that software architectures are so often mediocre is that architecture frequently takes a back seat to more mundane concerns such as cost, time-to-market, and programmer skill.

Again, we need some efforts from the upper echelons of your organization if there is to be merely a semblance of progress; here is a message for your managers:

If you think good architecture is expensive, try bad architecture.

The need for collaboration between management and developers appears most clearly when taking into account Conway’s law and its direct impact in the actual architecture of the final product. This law is an empirical observation made by Melvin Conway in annus mirabilis 1967 that has been confirmed by research, again, again, and again, and consists of a simple rule: no matter what you do, your software architecture always mimics your organizational chart:

Frequently, the organization of the system reflects the sprawl and history of the organization that built it (as per CONWAY’S LAW) and the compromises that were made along the way.

The “Big Ball of Mud” paper goes on to describe what such an “architecture” looks like in real life, and the picture is not pretty.

All too many of our software systems are, architecturally, little more than shantytowns.
(…)
What does this muddy code look like to the programmers in the trenches who must confront it? Data structures may be haphazardly constructed, or even next to non-existent. Everything talks to everything else. Every shred of important state data may be global.

What can be done, then? Should we sit down and plan our architecture beforehand? Well, yes, but with caveats:

One thing that isn’t the answer is rigid, totalitarian, top-down design.
(…)
Kent Beck has observed that the way to build software is to: Make it work. Make it right. Make it fast.

Thanks, Kent. The issues behind your software project are not technical, but of human nature:

It is interesting to ask whether some of the differences in productivity seen between hyper-productive organizations and typical shops are due not to differences in talent, but differences in terrain.

You might want to use a 4x4 vehicle when driving those muddy roads. Please note, however, the ever present toxic element of a macho-driven culture among 25 to 35-year-old Caucasian male brogrammers who will be very happy to create a Big Ball of Mud on purpose, because testosterone:

Status in the programmer’s primate pecking order is often earned through ritual displays of cleverness, rather than through workman-like displays of simplicity and clarity. That which a culture glorifies will flourish.

Ouch. What can be done, then? Well, to begin with, humble collaboration, less hubris:

One of mud’s most effective enemies is sunshine. Subjecting convoluted code to scrutiny can set the stage for its refactoring, repair, and rehabilitation. Code reviews are one mechanism one can use to expose code to daylight.
(…)
Another is the Extreme Programming practice of pair programming.

Thanks again, Kent. Do not forget to educate your customer to avoid the dreaded MDD (Market-Driven Development) strategy.

When you build a prototype, there is always the risk that someone will say “that’s good enough, ship it”.
(…)
Alexander has noted that our mortgage and capital expenditure policies make large sums of money available up front, but do nothing to provide resources for maintenance, improvement, and evolution.

Even though, history shows that some instances of a Big Ball of Mud actually became Earth-shattering events; these events are very rare, however:

The Wiki-Web code at www.c2.com also started as a CGI experiment undertaken by Ward Cunningham also succeeded beyond the author’s expectations. The name “wiki” is one of Ward’s personal jokes, having been taken from a Hawaiian word for “quick” that the author had seen on an airport van on a vacation in Hawaii.

We need an equilibrium, then, between heavy upfront design and agile flexibility. Maybe, just maybe, overcoming the aversion to writing common in so many software developers might be a starting point.

The waterfall approach arose in response to this muddy morass. While the code-and-fix approach might have been suitable for small jobs, it did not scale well. As software became more complex, it would not do to simply gather a room full of programmers together and tell them to go forth and code. Larger projects demanded better planning and coordination.
(…)
You can’t simply aspire to solve the problem at hand once and for all, because, by the time you’re done, the problem will have changed out from underneath you.
(…)
When designers are faced with a choice between building something elegant from the ground up, or undermining the architecture of the existing system to quickly address a problem, architecture usually loses.
(…)
If you can adapt quickly to change, predicting it becomes far less crucial.

Bingo.

This article was written during an era of tremendous change in the field of programming methodology, culminating with the publication of the Agile Manifesto in 2001. Some ideas in the “Big Ball of Mud” article remind us of concepts that have, thankfully, become standard in 2024, like CI/CD:

Microsoft mandates that a DAILY BUILD of each product be performed at the end of each working day.

Yet, somebody has to write (and maintain!) those YAML pipeline definitions, and that means we need to budget for that. Since we are doing the numbers, could you please raise a bit more venture capital to cover for documentation costs?

The eternal, Zen-like flow of change must be channeled, like a river toward a dam, in order to leverage its power.

Software systems cannot stand still. Software is often called upon to bear the brunt of changing requirements, because, being as that it is made of bits, it can change.
(…)
Therefore, factor your system so that artifacts that change at similar rates are together.

One of the major tenets exposed in the book “Head First Object-Oriented Analysis and Design” is to “separate what changes from what does not”. Unsurprisingly, this idea surfaces here as well:

The first step on the road to architectural integrity can be to identify the disordered parts of the system, and isolate them from the rest of it.

Finally, do not be afraid of rewriting your application if all else fails. Yes, it should be the last resort, but one that should be considered anyway; remember to avoid doing it for the wrong reasons or just for the sake of hype, however:

Sometimes it’s just easier to throw a system away, and start over.

The conclusion of this article should make us finally accept the Big Ball of Mud, not as an “Anti-Pattern” but instead as the defining structure of all software, the most probable outcome, the final state to which all code flows, and the unavoidable thermodynamic state of every single software project ever conceived.

However, its undeniable popularity leads to the inexorable conclusion that it is a pattern in its own right. It is certainly a pervasive, recurring solution to the problem of producing a working system in the context of software development.
(…)
If this makes muddiness a frequently terminal condition, is this really a bad thing? Or is it a blessing that these sclerotic systems yield the stage to more agile successors?
(…)
People build BIG BALLS OF MUD because they work.
(…)
It is not our purpose to condemn BIG BALLS OF MUD. Casual architecture is natural during the early stages of a system’s evolution. The reader must surely suspect, however, that our hope is that we can aspire to do better.

At the risk of stating the obvious, Brian Foote’s article is timeless, but more important, it also represents a deliverance. Aspire to do better, but be aware that if you cannot, it is not your impostor syndrome kicking in, or because you are doing it wrong (whatever it is).

Paraphrasing Jerry Lee Lewis,

You shake my nerves and you rattle my brain
Too much love drives a man insane
You broke my will
But what a thrill
Goodness gracious, great balls of mud!

Cover photo by Aubrey Odom on Unsplash.

Continue reading Dylan Beattie or go back to Issue 069: Architecture. Did you like this article? Consider subscribing to our newsletter or contributing to the sustainability of this magazine. Thanks!
Back to top