At the risk of alienating most of the readership of this magazine, here is a confession. I hated Python for a very long time. My issue was not with the language per se, even though the indentation rules have put me off for a while. No, the reason I kept myself away from Python was the unfortunate contact with some (too many for my taste) hubris-filled Python developers. Well, that, and the no man’s land that was the transition between Python 2 and 3.
That was, of course, unfortunate. But after a few years, pragmatism naturally brought me back to Python again; not so much the language per se, which I now neither like nor dislike, but the sheer amount of libraries available (and their excellent quality). Another big contributing factor to this renewed interest in Python was the long awaited announcement of the deprecation of Python 2; this event signals a major point of maturity in the language, and one that was badly required.
The ecosystem around Python is outstanding, and the rise of Machine Learning cemented its status as the new Fortran, while at the same time it became the new pseudocode, or even, the new BASIC.
These days it is not only hard, but unwise for a developer to ignore Python. This article is the product of six months of exploration in online and offline media, to find out how much Python has permeated our craft. The result was a surprise even to this author. Python is not only everywhere, it has become a “default” programming language for the computer industry.
The Ecosystem
As stated above, the first thing that brought me into Python was its libraries. I needed to get stuff done (who does not?) and Python had precisely the libraries I needed. Its standard csv package was the first one I explored and used with intensity, almost 20 years ago; next came argparse, doctest, and shutil, all part of the standard library.
Manipulating data in CSV format took me to export it to other formats; Python-OpenXML provided a first solution, later enhanced with Pandas to manipulate Excel files.
Little by little, I started to use Python more and more. At some point I got a job as a Django application developer, a framework pompously marketed “for perfectionists with deadlines”; a punchline filled with hubris and undeserved merit. Thankfully these days Cloud Native Python apps are built around less monolithic options such as Flask, Pyramid, or Tornado.
In terms of IDEs, PyCharm, Spyder, or Wing, are fighting for the attention of developers, many of whom are migrating to Visual Studio Code with Microsoft’s own Python extension (who by the way are very interested in the Python ecosystem lately).
Developers might be interested to use mypy, Flake8 or Pylint to check the quality of their code.
Most Python scripts start and end their life in the command line. But for those interested in providing a GUI, choices abound: Kivy for touchscreen apps, the venerable wxPython, the powerful Qt for Python, and its often overlooked sibling PySide2. In the experimental part of the spectrum, REMi, and Gooey deserve special mentions; the former turning scripts into fully-fledged web apps, and the latter transforming any CLI script into a desktop GUI application.
Python has also strong interoperability with other languages and runtimes. Suffice to mention SWIG and Boost.Python, enabling interoperability with C and C++; and Jython, enabling its use in the JVM–still lacking support for Python 3 at the time of this writing.
Speaking about alternative implementations of the language, one must also mention PyPy, recommended by Guido van Rossum for its speed, and IronPython for .NET, whose Python 3 alpha implementation was recently released.
Python is a fertile ground for package managers and build systems: Conan, Poetry, Wheels, redo (a recursive build system), and of course the venerable pip, together with the more recent pipx (an equivalent to JavaScript’s npx command).
Many ground-breaking ideas have been powered by Python lately: Ansible; asciinema; the Atheris code coverage tool; authentik (replacement for Keycloak); Beautiful Soup to parse HTML; bump2version; Celery for distributed queue processing; Click and Typer to replace argparse; Cookiecutter; Depix to recover passwords from pixelized screenshots; Diagrams; Ecco to explore NLP models; Glances to monitor your servers; GOMP to compare Git branches; IceCream; jrnl to write your journals; lorem to create placeholder text; Outrun to run commands in a separate machine; Pillow to manipulate images; pdoc to generate API docs; PyGithub; PyYAML; qrcode; rainbowstream; Ramanujan Machine; Requests for your HTTP needs; Rich for formatting text in the console; TOML because there are not enough configuration languages; Vulture to find dead code; youtube-dl to download videos online; Weechat Matrix protocol client; whereami; the Whoosh search library; and so much more.
If Rust seems to be the language of the great rewrite, Python is definitely the language of the first release.
Machine Learning and Scientific Computing
In the past decade, Python single-handledy replaced Fortran as the programming language of choice for scientific computing. It all started with Numpy, and then followed with Jupyter notebooks, a concept loosely based on Knuth’s idea of literate programming. Followed Anaconda, SciPy, PyTorch, and the rest is history. In particular, the rise in popularity of Machine Learning owes a great deal to the pre-existence of these tools.
Getting Python
Installing Python is still a non trivial task, and usually the biggest hurdle for developers new to the ecosystem, particularly given the sheer number of contradicting advice online. May this section remove some of the confusion, through proven techniques this author has used to install Python in different environments.
Windows users can simply use Chocolatey to get their Python environment installed, and ready to use. Mac users are better served by not using Homebrew for it; the author of this text heartily recommends using pyenv instead for both Mac and Linux users.
No matter which technique you use to install Python, always use a virtual environment per project. This ensures they are self contained, with well-defined dependencies, and are guaranteed to work in other people’s computers.
$ python3 -m venv .venv
$ source .venv/bin/activate
$ pip install PyYAML
$ pip install PyGithub
$ pip freeze > requirements.txt
# Later on...
$ pip install -r requirements.txtVirtual environments also help developers get their code inside Docker containers easily; remember to pass the --no-cache-dir option to pip install in your Dockerfiles. Never forget to add a proper .gitignore file to your project.
Learning Python
There is no shortage of material to learn Python. The following books might serve as a starting point, in the order specified:
- Python Programming: An Introduction to Computer Science by John M. Zelle, Ph.D.
- The Python Guide for Beginners by Renan Moura.
- Effective Python by Brett Slatkin.
- Beej’s Guide to Python Programming by Brian “Beej Jorgensen” Hall.
- Business Python by Theodore Deden.
- Create GUI Applications with Python & Qt5–PyQt5 Edition and PySide2 Edition by Martin Fitzpatrick.
If you are interested in online tutorials, I will only mention here Python basics and Real Python, but a short DuckDuckGo search will bring many more resources.
Guido van Rossum
The creator of Python, also known as its “Benevolent Dictator for Life”, has had quite an illustrious career, closely monitored by the press. In the past 15 years he spent his time between Google and Dropbox. In the former he contributed to the creation of the Mondrian code review tool, written in 2006 to replace a mail-based workflow. Later on, he arguably kicked-off the first popular PaaS, Google AppEngine, whose first supported runtime was, of course, Python.
A curious fact: Python had not yet been featured in the HOPL conferences created by Jean Sammet. Interestingly enough, Mr. Van Rossum started working in such paper in 2006, to be presented in HOPL III, but after much back and forth, he used the material to start a new blog about the history of Python instead.
After Google, Mr. Van Rossum went to Dropbox, from 2013 to his retirement in October 2019. At the time of this writing he is said to be working in the mypy static type checker–or at least, that is what the project blog says.
More
Of course, not everything made with Python was successful. Suffice to mention the GadflyB5 SQL relational database, or Mercurial; excellent tools in their own right, but shadowed by much more popular options. reStructuredText also is generally dismissed, developers usually prefering Markdown or Asciidoc.
Still, Python became endless material for memes that make developers laugh. There is a Zen of Python. Python made developers sing. The Texas Instrument TI-84 Plus CE-T Python Edition calculator can edit and run Python code. People use Python inside Excel spreadsheets instead of VBA. They create Gimp plugins or render movies with Python. They write serverless applications with it to process 558K transactions in 5 minutes. It was named the TIOBE Programming Language of the Year in 2007, 2010, 2018 and 2020. Its syntax has been translated from English to German, Chinese, Javanese, Lithuanian, Spanish, Swedish, and Russian, among others. People ask themselves WTF Python.
Even though I do not hate Python anymore, I still facepalm every time I type quit in the REPL and get the tone-deaf "Use quit() or Ctrl-D (i.e. EOF) to exit" message instead.
Cover photo by Nick Fewings on Unsplash.
