My Python Development Environment in 2021

Choosing which tools to use in your Python development environment might be the toughest part of Python programming. The Python tooling ecosystem continues to evolve rapidly. I’ll share what I need my tools to do, what tools are a part of my development environment, and why I use each one.

My Requirements

I develop web applications as my day job, specifically a Django application. I also maintain some open source packages and want to do more open source in the future. I use Ubuntu for my development operating system and avoid the system’s Python interpreter like the plague. In my younger years, I bricked my computer messing with the system interpreter. I love easy-to-use tools that improve code quality such as Flake8 and Black and want them to be seamlessly integrated into my development flow.

I want my tools to help:

• Install and switch between multiple Python interpreter versions while avoiding the system interpreter.
• Install and manage the version of other languages’ package managers and run times such as Node.js.
• Make Python CLI utilities globally available without using the system interpreter or having to switch between virtual environments.
• Simplify publishing of Python packages because I don’t want to mess with a setup.py file.
• Manage numerous Python virtual environments.
• Run and maintain linters and formatters for multiple languages.

The Tools

Asdf

Asdf is a CLI tool that can manage multiple language runtime versions on a per-project basis. Initially, I just used Pyenv to manage my Python versions, but on some projects, I need to manage my Node.js version as well. I started using Nvm until a colleague showed me Asdf. The naming isn’t best, but naming things is hard. Other than the name, the tool rocks and having a single tool to manage all of my runtime versions simplifies my development environment. On top of that, Asdf’s Python plugin uses Pyenv to install Python versions and integrates Pyenv’s config formats which eased my transition.

Example usage

• Install Asdf
• Add the Python plugin

asdf plugin add python

• Install your favorite Python version

asdf install python 3.9.6

• Set it globally

asdf global python 3.9.6

• Validate your favorite version is global

python --version

If you, like me, are forced to work with JavaScript.

• Add the Node.js plugin

asdf plugin add nodejs

• Install the latest LTS version because the JavaScript ecosystem is volatile

asdf install nodejs lts

• Set the LTS version globally

asdf global nodejs lts

• Validate it

node --version

Pipx

Pipx helps you install and run CLI tools written in Python within isolated environments. I use Python CLI tools such as Awscli and Pre-commit, and I definitely don’t want these tools installed to the system interpreter. Installing these tools to their own virtual environment and manually switching quickly becomes annoying and inconvenient. Pipx fixes this by managing the isolated virtual environments and making these CLI tools globally available.

Example usage

• Install pipx
• Use pipx to install Pre-commit

pipx install pre-commit

• Validate Pre-commit installed

pre-commit --version

Poetry

Poetry helps manage and install dependencies of Python projects and allows you to publish Python packages. Why not just use Pip to install and manage your Python dependencies? Poetry builds on top of Pip with some nice extra features.

Poetry enables:

• management of per-project virtual environments.
• switching between virtual environments with different Python versions.
• maintaining a top-level record of both production dependencies and development dependencies.
• publishing of Python packages.

Why is a top-level record of dependencies important? Pip freeze produces a file with all dependencies including dependencies of dependencies. I want to keep my project’s dependencies up-to-date and without this top-level record, updating packages becomes a struggle. With Poetry, I run poetry update, and I’m up to date. I use docker to deploy a lot of my projects, and image size always becomes an issue. Poetry separates production dependencies from development dependencies which helps reduce image sizes. I love Pytest, but it doesn’t need to be installed into my production containers.

Example usage

• Install Poetry
  • I recommend against installing with Pipx. For more info, read over this issue
• Start your new project

poetry init

• Add a new dependency

poetry add httpx

• Add a new development dependency

poetry add --dev pytest

• Build and publish your Python package to PYPI

poetry publish --build --username laactech --password super_secret

Pre-commit

Pre-commit is a framework for managing and maintaining multi-language git hooks. As I mentioned, I mainly work on a Django web application. I like to use git hooks to run code linters and formatters on each code commit. I maintain an up-to-date version of my Pre-commit config and use it on almost every project. I’m a green JavaScript developer and don’t know what conventions to follow. I like using ESLint and Prettier to keep me honest. With Pre-commit, I have one config file that defines how to use Black, Flake8, Prettier, and ESLint.

Example usage

• Install Pre-commit with Pipx

pipx install pre-commit

• Use my Pre-commit config or define your own
• Install the hook in your git repository

pre-commit install

• Run the hook across your entire code base

pre-commit run --all-files

Final Thoughts

For a beginner starting their Python journey, this setup is overkill. For me, all of these tools make my day-to-day development life easier. The learning curves aren’t too steep and investing the time pays off. The Python tooling ecosystem continues to change rapidly. Pipsi got abandoned in favor of Pipx. Poetry and Pipenv duke it out for Python packaging dominance while other tools continue to pop up. I hope that I’m not writing this same article two years from now with all new tools.