What, to you, is quality software?
I’m sure that I will get different responses from different people. Quality is not absolute, and what you consider high-quality may be irrelevant to somebody else. Your view of high quality may even be a sign of poor quality to another reviewer. Let’s take a look at some aspects of software quality.
Code Quality
Developers, myself included, often think of the quality of the software’s source code: is it easy to understand and change? Does the code use paradigms that I find admirable? Does it use functional programming, for example, if I’m in to that sort of thing?
Did the author write automated tests? Do the tests provide good coverage? Are they appropriately distributed between different levels of granularity? Do the “grains” (the units, modules, or subsystems) represent useful models or abstractions?
Is there documentation? If there is not, is it because the code is self-documenting? Is there a single command to type or button to press to get a production build out? Does committing to the repository automatically trigger a production build?
Does Code Quality Matter?
These questions all relate to a particular perspective on the quality of software, but we must be honest with ourselves. For the most part, customers considering whether to license a particular software product or subscribe to software as a service very rarely even look at the source code, let alone make an informed judgement as to its quality. In most cases, they aren’t even allowed to look.
Even in the case of free software or open source products, the likelihood of spy-before-you-try is low. I cannot bring to mind the thousands of such projects I must have used during my career to date: Eclipse, LibreOffice, Linux, the GNU Compiler Collection, Darwin, clang, LaTeX, Firefox, FreeBSD, GNU Emacs, node.js…and I can’t think how vanishingly small must be the fraction of those of which I have even partially examined the source code. Probably not even one per cent.
Lemons
Now that means that if the quality of a software product, or service, is related to the code quality, then there’s a lemon market for software. People can’t (or don’t) evaluate the quality of the code, so won’t pay more for premium-quality code (and hence software) than they would for a clunker.
(As an aside, the solution for the lemon problem in motor vehicles is to provide a warranty. In software, commercial and free software alike is provided “AS IS”, all in capital letters, with no suggestion that the maker expects it to work the way you like, or even the way they said in the marketing materials.)
Zen and the Art of Software Maintenance
The quality of the code in a software system is an aspect of its internal quality. Borrowing an analogy from Robert M. Pirsig, interest in this form of quality is like being interested in the internals and maintainability of a motorcycle engine. It’s a “classical” form of quality, in which one must be mindful of all the workings and details of the system under consideration.
Other people will have other ideas of software quality, perhaps informed by the “romantic” school of quality.
To Accessibility and Beyond
Is the product accessible? That may mean that the controls are suited to full keyboard navigation, text-to-speech readers, joystick control, or other adaptive input and output schemes. Accessibility also includes ensuring that the colour scheme used through the UI is suitable for people with colour vision deficiency.
Some would say that accessibility is one aspect of the wider principle of usability and consideration of the user experience. Can the capabilities of the software be discovered easily? Can they be used efficiently? Do the capabilities that a given person requires match their expectations? Do those capabilities even exist?
Other External Qualities
We could carry on listing other external “qualities” of a software product or service: security is a very big topic and indeed the next issue of de Programmatica Ipsum will focus exclusively on that.
Performance, both in terms of speed of computation and responsiveness (a researcher running their scientific codes on a high-performance computing installation will experience very high latency, as the queueing system could take days to even start their job; they may not think of this as “unresponsive” if submitting to the queue happens quickly). The resource usage is another form of quality, as is the clarity of the customer documentation or even the helpfulness of the customer support team.
How We Plan For These Qualities
Many of these quality attributes will be described by most people on a software project as the “non-functional requirements” and batched together as things we should probably keep an eye on while planning to construct the functional requirements. Meanwhile, the software architect (if there is one) will be trying to design a system that satisfies all of the non-functional requirements while paying half a mind to the question of whether it can satisfy the functional requirements.
Which finally, nearly eight hundred words into an article on software quality, brings us to the elephant in the room: does the software even do the things people want of it? That seems like an important measurement of software quality, too.
The Quality of Correctness
Does our software do what our customers want, or need? Manny Lehman tells us that in many cases the answer to this question is not stable. You are not making it up and your customers are not deliberately being awkward, their needs genuinely are evolving. Perhaps, in a very postmodern way, the trigger for their evolution was the introduction of the software itself. The availability of version 1 freed up some time, so that they realised they could do these other things if only they had version 2.
Given the evolutionary nature of software requirements, how should we consider the “quality assurance” function in a software team? For a start, let us rid ourselves of a misconception: your “QAs” do not assure quality. At best, they measure quality by testing (or creating automated test scripts for) your software. But testing does not assure us of quality. As Dijkstra said, it can tell us when bugs are present, but not when they are absent.
Regression Tests
Most of the software projects I’ve worked on have had a battery of “regression tests”. Only some of these were written to detect regressions. Most were written while TDDing, or to demonstrate that a feature worked as part of that feature’s construction: they are accretion tests, rather than regression tests.
A true regression test would be one where we know that the behaviour must not be reverted. But that doesn’t mean that we should test anything that was ever added, it should mean knowing what your customers are using today and how bad it would be if it were to break.
“Enough” Quality
Notice that I said how bad, as there are different amounts of badness. The Joel Test asked whether we always fix new bugs before we write new code. That doesn’t seem like a good way of working. Success in business is always about opportunity, and so we need to consider opportunity cost. If three people would be put off our product by a minor bug, and thirty people would buy the product if we added a minor feature, we should write the new code before fixing the bug. On the other hand if the bug breaks every day features for every customer of our product, then yes, it’s time to fix it.
There are lots of ways to evaluate the quality of a software product. Now what, to your customers, is quality software?
—
Photo by Dane Deaner on Unsplash.
