Have you ever asked yourself why do we live in society, surrounded by others? Why have humans gathered in slowly-growing communities at the dawn of our civilization? It turns out that humans are terrible when it comes at surviving alone. We are not particularly strong or fast, which means that we can succumb at the attack of any predator. We cannot fly by flapping our arms, we cannot swim at great speed in the sea. Our bodies suffer plenty of ailments and are prone to quite a few lethal illnesses. To add insult to injury, we carry a big, complex, and fragile organ inside our skulls that requires quite a few calories a day to function properly. Even worse, said organ is supposedly at the origin of a psyche (a novel concept in nature, apparently) with an annoying tendency towards anger, fear, and depression.
So, get together we do. Turns out, there were several advantages in doing so; to begin with, work specialization is a huge one. Some people are better at hunting, some others at growing vegetables, some others at teaching, some others at making clothes, some others at invoking the spirits of the forest. Getting together meant that we could be more than just the sum of the members of our communities. We could have more things. We could learn from others. And, yes, we could survive better.
All of this became possible because, at some point around a dozen millennia ago, we decided to trust each other.
Our modern society, however, seems to have forgotten this basic principle. We forgot that we were able to use empathy as a superpower, and through the simple act of taking care of one another, to build real webs of trust. Instead, human communities of various scales are crippled today with decaying infrastructure, rampant corruption, abysmal crime statistics, lack of professional, economical, and educational opportunities, war, political or religious prosecution, or even worse, with full-blown, state-driven, and technologically-enabled annihilation.
Faced with such a situation, many human beings (among which you can include the author of these words) decided to just pack up and move to other valleys, where the grass is supposedly greener; not all can afford or are allowed to do so, however. On the extreme opposite side of this spectrum, billionaires on a quest against empathy plan on polluting colonizing other planets to escape the mayhem they have created. Somewhere in the middle, the 0.1% of mankind who cannot (yet) afford a spaceship orchestrate their retreats to secure locations ranging from a private island in the Pacific Ocean, to Dubai, or to a secluded and tax-friendly ski resort in the Swiss Alps.
As a result, we are all either fleeing our society, or dreaming of doing it someday. So much for trusting each other. Even worse, this inconvenient feeling named mistrust has translated to each facet of our human experience, including, as you can imagine, our software.
Tim Burton’s “Batman” movie features a scene where the Joker, played by the immense Jack Nicholson, and with Prince’s music in the background, is throwing dollar bills to the same Gotham City crowd he is planning to kill en masse. He then asks the trillion-dollar question: “Who do you trust?”
As a testimony to the brilliance of Burton, it is hard not to see in this short clip an allegory of our current world.
No Small Vulnerabilities
During the last decade, a troublesome one if we compare it to not-so-recent history, we can enumerate quite a few occasions in which software dropped its status from the pinnacle of human creativity, to the bottomless pit of the most abject hatred for mankind. And we, software workers and software practitioners, are the only ones to blame for this.
Let us recap a few highlights, beginning with 2015, when Volkswagen was caught tweaking its own diesel car software to cheat during emission tests. Somebody wrote, tested, and deployed that code.
Then during the same year, Uber engineers inserted special instructions in their apps to evade law enforcement in cities where it faced regulatory issues, identifying officials and preventing them from hailing Uber rides. Somebody wrote, tested, and deployed that code.
Boeing’s faulty (and even non-documented) Maneuvering Characteristics Augmentation System software caused two deadly crashes (Lion Air Flight 610 in 2018, and Ethiopian Airlines Flight 302 in 2019), killing 346 people in total. Somebody wrote, tested, and deployed that code.
From 2017 to 2020, Apple intentionally slowed down older iPhones via software updates, without informing users, to push them toward buying new models. By the way, you might want to collect the 20 USD that Apple owes you because of a privacy lawsuit settlement related to Siri early this year. Somebody wrote, tested, and deployed that code.
Speaking about Apple, let us not forget about the aptly named trustd apocalypse which prevented macOS Big Sur users of opening third-party applications in November 2020.
That same year, allegedly Russian hackers compromised software made by a company named SolarWinds, inserting backdoors that effectively worked against numerous government agencies and private companies (effectively for the hackers, that is). According to the SEC filing by SolarWinds, this supply chain attack affected approximately 18'000 organizations. We will never know the real number, and I do not think we really want to know it.
And let us not even get started on the Facebook and Cambridge Analytica affair, shall we. As we publish this article, we are still living the consequences of Mark Zuckerberg’s childish inability and unwillingness to accept the rules of a trusting society. Instead, he prefers to prevent the release of a book that candidly reveals the inner workings of a truly psychotic and untrustable organization, ruled by psychotic and untrustable billionaires.
What about exploits? They have become a yearly tradition like having birthday cakes; take any piece of software (open-source or not) or any hardware gizmo, ask a few security experts to scrutinize it for a while, and then give a fancy name to whatever vulnerability they find. Thus were born Heartbleed and FLUSH+RELOAD in 2014, ARMageddon and DRAMA in 2016, Spectre and Meltdown in 2018, PACMAN in 2022, Downfall in 2023, and ZenHammer in 2024.
We are still waiting for the winner of 2025. These exploits have become such a common fixture in our daily life that they not only feature a USENIX presentation or a paper on arXiv, but even their own website with their own domain name, with a nice logo or plush mascot designed specially for them. If you cannot trust our software, at least support us on Patreon or buy our swag.
No Plausible Deniability
The word “trust” has been very much abused through the years. These days we have the Zero-Trust Architecture, and a journalists’ association called “The Trust Project”. There are “antitrust lawsuits” which suggests the existence of a business meaning for the word “trust”. During the late 1950s Johnny Carson presented a TV show called “Who Do You Trust?”; and yeah, all US dollar bills feature the well-known “In God We Trust” motto.
Never mind that Sibert, Porras, and Lindell had warned us, already in 1995, that the Intel 80x86 processor architecture was not to be trusted from a security standpoint. Apparently Bill Gates forgot to read that paper, and then five years late he had to come up with a thing he called “Trustworthy Computing” because Back Orifice was undermining trust in his valuable Windows operating system.
Thankfully, Joanna Rutkowska did read Sibert, Porras, and Lindell’s paper, and included a severe and cold-blooded definition of the concept of trust in a 2015 paper called “Intel x86 considered harmful”. Sensitive souls beware:
The word “trusted” is a sneaky and confusing term: many people get a warm fuzzy feeling when they read it, and it is treated as a good thing. In fact the opposite is true. Anything that is “trusted” is a potentially lethal enemy of any secure system. Any component that we (are forced to) consider “trusted” is an ideal candidate to compromise the whole system, should this trusted component turn out to be buggy or backdoored. That property (i.e. the ability to destroy the system’s security) is in fact the definition of the term “trusted”.
An IBM engineer had said already in the 1970s that a computer could never be held accountable, so I do not see what is so surprising here; particularly now that the UK government is weakening safety worldwide with its actions, the current software crisis is, just like most economic crisis, one caused by mistrust.
Ken Thompson, of Unix, C, UTF-8, and Go fame, declared during his 1983 ACM Turing Award speech called “Reflections on Trusting Trust” that
You can’t trust code that you did not totally create yourself. (Especially code from companies that employ people like me.) No amount of source-level verification or scrutiny will protect you from using untrusted code. In demonstrating the possibility of this kind of attack, I picked on the C compiler. I could have picked on any program-handling program such as an assembler, a loader, or even hardware microcode. As the level of program gets lower, these bugs will be harder and harder to detect. A well-installed microcode bug will be almost impossible to detect.
These words seem oddly prophetic when we remember the XZ Utils backdoor discovered merely one year ago. Remember to thank Andres Freund for performing what Ken Thompson considered almost impossible 40 years ago.
We proposed an oath for software developers a few years ago, which included the following statement that we think perfectly summarizes our feelings in the matter of trust:
I will remember that I do not merely create a system or implement an algorithm, but I create systems for the highest benefit of society, who will have to use it and who will store their most confidential information within. My responsibility includes these related problems, if I am to solve adequately the problem at hand.
Nothing Left For Me To Do
Let us quote Sting to finish this admittedly depressing article:
You could say I lost my faith in science and progress
You could say I lost my belief in the Holy Church
You could say I lost my sense of direction
And you could say all of this and worse but
If I ever lose my faith in you
There’d be nothing left for me to do
Empathy and trust in one another are the fragile glue that keeps our society in an ever-fragile equilibrium. We must acknowledge the fact that there is no gizmo, AI, protocol, library, programming language (no, not even Rust), platform, or mobile application to replace them, and in this acknowledgement resides our chance for survival as a species. When it comes to software, the answer to the question “Who Do You Trust?” should be a strong, resounding, and unanimous voice saying: “all of us”.
In other words, TL;DR: Please give a shit.
Cover photo by Azzedine Rouichi on Unsplash.