Python

Deep dive

Decision table

Memory and performance model

• CPython: reference counting (immediate frees) plus a cycle collector; __slots__ shrinks per-object memory; arrays of numbers belong in NumPy (8 bytes per float64, contiguous).
• 3.11 brought order of 25 % speedups via specializing adaptive interpreter; an experimental JIT exists since 3.13 (PEP 744), gains still modest.
• Function calls and attribute lookups are costly; hoist them out of hot loops, or better, remove the loop.
• Startup order of 10-50 ms bare; large imports (torch, pandas) add hundreds of ms to seconds.

Concurrency model

• Threads: real OS threads, but with the GIL only one executes bytecode; fine for I/O, and for native code that releases the GIL (NumPy, many C libraries).
• asyncio: single-threaded event loop with async/await; one blocking call stalls everything.
• Processes: true parallelism, at the cost of pickling data between them; use shared memory or Arrow for large arrays.
• Free-threaded builds (python3.14t): real parallel threads; single-thread speed is somewhat lower and C extensions must declare support.

Code

Python 3.11+: structured async I/O, CPU work in processes, and why vectorization matters.

import asyncio
import time
from concurrent.futures import ProcessPoolExecutor

import numpy as np


async def fetch(i: int) -> int:
    await asyncio.sleep(0.1)  # stands in for a network call
    return i * i


async def gather_all() -> int:
    async with asyncio.timeout(2):
        async with asyncio.TaskGroup() as tg:  # one failure cancels the rest
            tasks = [tg.create_task(fetch(i)) for i in range(100)]
    return sum(t.result() for t in tasks)  # about 0.1 s total, not 10 s


def cpu_heavy(n: int) -> int:
    return sum(i * i for i in range(n))


def py_dot(a: list[float], b: list[float]) -> float:
    return sum(x * y for x, y in zip(a, b))


if __name__ == "__main__":  # required for process pools on macOS/Windows (spawn)
    print("async:", asyncio.run(gather_all()))

    with ProcessPoolExecutor() as pool:  # parallel across cores despite the GIL
        print("cpu:", sum(pool.map(cpu_heavy, [2_000_000] * 8)))

    n = 1_000_000
    a, b = np.random.rand(n), np.random.rand(n)
    la, lb = a.tolist(), b.tolist()
    t = time.perf_counter(); py_dot(la, lb); t_py = time.perf_counter() - t
    t = time.perf_counter(); float(a @ b); t_np = time.perf_counter() - t
    print(f"pure Python {t_py * 1e3:.0f} ms vs NumPy {t_np * 1e3:.2f} ms")  # order of 100x

pyproject.toml with uv:

[project]
name = "svc"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = ["fastapi>=0.115", "uvicorn[standard]>=0.30", "numpy>=2"]

[dependency-groups]
dev = ["pytest>=8", "ruff>=0.6", "mypy>=1.11"]

uv sync --locked          # exact versions from uv.lock into .venv
uv run pytest
uv run uvicorn svc.main:app --workers 4   # one process per core for CPU headroom

Efficiency

• Cost: CPython boxes every value: an int is 28 bytes (measured with sys.getsizeof(1) on 64-bit CPython), and pure-Python loops run order of 10-100x slower than C. Python 3.11 is 10-60 % faster than 3.10, 1.25x on average (What's New 3.11). Bound: interpreter dispatch and allocation for pure Python; memory bandwidth once the work sits in NumPy or Polars.
• Levers:
  • vectorize into NumPy, Polars or PyTorch — loops run in C (order of 10-100x) — array and table work
  • move to Python 3.11 or newer — 1.25x on average, 10-60 % range (What's New 3.11) — any pure-Python code
  • free-threaded build — threads run bytecode in parallel; single-thread overhead about 1 % (macOS aarch64) to 8 % (x86-64 Linux) (free-threading HOWTO) — CPU-bound threaded code
  • Rust or C extension via PyO3/maturin — native speed for the loop that will not vectorize (order of 10-100x) — hot loops
  • uv for installs — 10-100x faster than pip (uv docs) — CI and container builds
• Naive → standard → state of the art:
  • Python loop over rows · slowest, order of 10-100x off C
  • vectorized NumPy/Polars call · native loop plus one call overhead
  • native extension or GPU kernel behind a Python API · native speed; Python only orchestrates

Build and deploy

• Containers: python:3.x-slim base, uv sync --locked --no-dev, non-root user; wheels for native deps (manylinux) avoid compilers in the image.
• Web servers: uvicorn/gunicorn with several worker processes per container; size by cores and memory per worker.
• Single-file tools: uv run script.py with inline script metadata (PEP 723) declaring dependencies.
• Type-check in CI; Python's lack of compile step makes tests and types the only safety net.

Pitfalls

• Mutable default arguments (def f(x=[])) shared across calls.
• Blocking calls (requests, time.sleep, CPU loops) inside async def handlers.
• Expecting threads to speed up CPU-bound pure Python on the default build.
• Row-by-row loops over pandas DataFrames (iterrows): use vectorized ops or Polars expressions.
• pickle/joblib/torch.load(weights_only=False) on untrusted files: arbitrary code execution.
• Global interpreter-wide state in imports (side effects at import time) breaking multiprocessing or tests.
• Unpinned dependencies; installing into the system Python.
• Late-binding closures in loops (lambda: i captures the variable, not the value).

Implementations

Sources

• Python Software Foundation, The Python Language Reference. docs · asyncio documentation. docs
• Python Software Foundation, What's New In Python 3.11 and Python support for free threading. docs · docs
• Gross, PEP 703: Making the Global Interpreter Lock Optional in CPython, 2023. spec
• Bucher, PEP 744: JIT Compilation, 2024. spec
• Astral, uv documentation. docs
• Harris et al., Array programming with NumPy, Nature, 2020. paper
• NumPy project, Case studies. site