Around 2004 I was working as a .NET software developer, building custom applications for a rather large Swiss customer who shall remain nameless. As part of the engagement, we had to not only write said software application following their requirements, but also deliver it in person. Yes, in person. We had to take a snapshot of the source code of the application (stored in a Visual SourceSafe repository, please do not laugh too loud), save it in a USB thumb drive, and drive to the location, where we would explain the deployment team how to build and deploy it.
Never mind that Visual Studio did actually offer the possibility of creating “MSI” installers for .NET applications, bundling precisely every single dependency together with the EXEs and DLLs, and copying every artifact in the precise location they needed to be for IIS (Windows’ canonical web server at the time) to find and run them. This customer did not want to hear about MSIs, and insisted on a manual delivery and a manual installation performed by their own team members.
Clearly, we were very far away from DevOps at the time, let alone basic automation, or standard common sense.
Of course, no matter how long the README or how well we explained the processes in person, the teams could not get it to work, and lots of blaming and finger pointing ensued. We repeated this operation for weeks, day after day, until one day, seemingly out of a miracle, we could get the application to run in their Windows Server 2003 boxes.
Insert hallelujah emoji here.
(For the record, it was a canonical “two-tier” monolithic application built on top of ASP.NET, with Visual Basic .NET, consisting of at most a couple of DLLs, talking to a rather dull SQL Server 2000 database behind the scenes. Seriously, nothing to phone home about.)
This scenario happened merely 22 years ago. Delivering software was not trivial, although I admit that the example I described above was one of the worst memories I have of this kind. But it was always complicated, requiring a separate workflow for every customer, and there was no hint of a standardized solution in the horizon.
Fast-forward 9 years, to June 7th, 2013. That day, in Paris, took place the dotScale 2013 conference, which still exists at the time of this writing, albeit named dotConferences. That day, a young French-American entrepreneur called Solomon Hykes from a company called dotCloud (a name unrelated to the conference despite the naming convention) delivered a 20-minute-long presentation that would have an unprecedented impact in our industry. And the recording of that session is, precisely, this month’s Vidéothèque entry: “Why we built Docker”
Mr. Hykes provides, in those 20 minutes, the essential idea behind containers, and the major issue they solved. Precisely, the problem I described from 2004: how to ship software from environment to environment, from workstation to workstation, from here to there, ensuring that every piece of your architecture remains in place and working.
If moving around a two-tier monolith was complicated in 2004, imagine moving around a microservices-based application, with many OLTP databases plus a few OLAP services, a key-value store for session management, a Kafka cluster for messaging, a stateless business logic engine written in some esoteric language, a UI built with some kind of magic JavaScript framework, connectors to external APIs, and who knows what else.
Solomon rightfully says that moving software from environment to environment is brittle and labor-intensive. I can confirm. Been there, done that. All of this was getting worse because of the explosion of cloud computing back in the 2010s.
And let us not even get into the burden of maintenance, applying security patches, und so weiter.
His idea, then, was to pay attention to the shipping industry, and the solution it came up with for moving stuff around: intermodal containers. Essentially, a fixed, standardized box that can be moved from a boat to a train, from a train to a truck, from a truck to a warehouse, regardless of the contents of the box, in a simple sweeping movement of a crane.
Borrowing from the lexicon of software developers, Solomon rightfully states that containers allow a separation of concerns: shipping companies do not care about what is in the box, as long as the box fits the correct sizing and weighing requirements (well, and hopefully as long as the contents are not illegal, either).
In contrast, Solomon states, the software situation is pretty embarrassing, and the current canonical solution at the time of the talk, virtual machines, are just plain inefficient, with their bundling of a kernel image for every instance of every application (even though utilities like Vagrant had substantially simplified their management up to that point).
Speaking about operating system kernels, the introduction of namespaces in the Linux kernel made containers possible, as well as the company that Solomon started (conveniently renamed later from dotCloud to Docker, Inc.).
Their flagship project (in version 0.4 at the time of the talk), simply called “Docker”, and built with the Go programming language, introduced three major breakthroughs: a standard format for container images; a set of tools for developers to build container images; and another set of tools for operators to run and manage those images. All in a lightweight bundle, seemingly integrated with whatever build and operation tooling already in place.
Solomon stayed at Docker Inc. until 2018, and these days he has been spotted starting other ventures, but apparently he does not do any AI stuff, so nobody is paying attention anymore.
Watch this month’s Vidéothèque article, “Why we built Docker”, by Solomon Hykes, on YouTube, a historically relevant keynote that had a terrific impact in our industry.
After watching this, we can recommend “Kubernetes: The Documentary” (first and second parts). If all of this sounds interesting for your next career move, check the “Getting Started with Kubernetes” video course by Nigel Poulton on the ACM website. And, if you feel adventurous, get your hands dirty with the classic “Kubernetes The Hard Way” by Kelsey Hightower. Welcome to the club.
Cover snapshot chosen by the author.