CPython's Compilation Pipeline
summary
Claim: CPython 3.13 refactors the compiler so Python code can observe and run intermediate stages, replacing the old four steps with source -- tokens -- AST -- optimized AST -- pseudo bytecode -- optimized pseudo bytecode -- bytecode -- code object. Tests for the peephole optimizer, code generator and assembler run as unit tests written in Python using _test_internal_capi, not C. A simple GUI called Codoscope visualizes the pipeline; the refactor already enables focused unit tests and clearer, less coupled compiler code.
AI-written summary · about
chapters (14)
- 0:00CPython compiler pipeline intro
- 1:09Python to bytecode compilation
- 2:54Existing compiler stages
- 4:23Motivation for refactoring
- 6:52Impact on codebase structure
- 9:01Performance driving changes
- 10:24Old vs new compiler stages
- 13:36AST to bytecode refactor
- 17:16Assembler to bytecode
- 19:25New stages via test_internal_capi
- 20:32Unit testing optimizations
- 23:01Visualizing compilation
- 23:44Summary and future
- 24:30Q&A
related talks
30mDemystifying Python’s Internals: Diving into CPython by implementing...
24mThe AST and Me
28mPipe-ifying Python: The how and why of interpreter hacking
27mPython: From Source to Execution
01h 03mA tour of CPython's runtime
36mHacking the Python AST
17mInside a PyPy JIT: is that even legal?
29mWhat they don't tell you about building a JIT compiler for CPython