How I write Python

HTTPX offers a similar API to Requests in both sync and async and with integrated type annotations. (There are third-party type annotations for Requests.) HTTPX makes it easy to use a SOCKS proxy.

urllib.request in the standard library works well as a fallback.

Selectolax is faster than Beautiful Soup with either html5-parser or html5lib. It is easier to install than html5-parser since it doesn’t require disabling binary wheels for lxml. The API design helps with type safety.

tomllib has been part of the standard library since Python 3.11. While configparser is also in the standard library, TOML is standardized and allows more structure. For writing TOML files, there is tomli-w.

Whenever correctly handles daylight saving time and uses types to prevent common mistakes. For more context, see “Ten Python datetime pitfalls, and what libraries are (not) doing about it” by Arie Bovenberg2024.

markdown-it-py follows the CommonMark standard, is typed, and is faster than other CommonMark libraries implemented in pure Python.

msgspec provides fast JSON, MessagePack, TOML, and YAML serialization of dataclass-like classes with type support.

Jinja offers expressive, flexible templates for HTML and other text formats. Its macros have proven particularly useful in practice.

yarl percent-decodes paths and has a well-designed API with properly named fields and query parameters stored in a MultiDict.

As a fallback, urllib.parse in the standard library can handle both parsing and quoting.

xmltodict is the most fun XML library I’ve used. Despite its name, xmltodict is bidirectional: it can both parse and generate XML.

I have a Copier template that packages my preferred development tooling. Type checkers are covered in the static-typing section.

uv is fast and manages Python versions as well as projects. It can also run single-file scripts with dependencies.

As a fallback, Poetry is capable and uses standard metadata since version 2. Poetry works on platforms where uv doesn’t, like NetBSD, though it has problems with PyTorch.

codespell checks for common misspellings in code. Ruff is a fast Python linter and code formatter written in Rust that can replace many other tools, like black, isort, pylint, and flake8.

shiv is a command line utility for building self-contained Python zipapps. Its only serious downside is that it only packages binary dependencies for the current platform.

pytest produces more readable tests with assert compared to the specialized assertions in unittest. It has a coverage plugin, pytest-cov.

As a fallback, unittest in the standard library isn’t bad.

Python has robust optional static typing. If you took a break from Python in the mid-2010s like I did, it may come as a surprise. This page links to resources for writing statically-typed Python, plus a few that make use of type hints at runtime.

  • jsonargparse—“Implement minimal boilerplate CLIs derived from type hints and parse from command line, config files and environment variables”
  • Typed Argument Parser (Tap)—“Typed argument parser for Python”
  • Typer—“Typer, build great CLIs. Easy to code. Based on Python type hints.”
  • tyro—“Zero-effort CLI interfaces & config objects, from types”

  • poltergeist—“Rust-like error handling in Python, with type-safety in mind.”

My preferred type checker is Pyright.

  • Fast
  • Good error messages