Demystifying Python’s Internals: Diving into CPython by implementing...
summary
Adds a real pipe operator to CPython by editing grammar and tokens, generating a new AST node in python.asdl, teaching the PEG parser how to emit it, changing the compiler to emit a BINARY_PIPE_CALL opcode, and extending the eval loop to call the function with the left value via the C API. Steps include regenerate the tokenizer (grammar/tokens → make regen), editing lib/opcode.py, recompiling CPython, and running tests. The prototype builds and runs, demonstrating source→AST→bytecode→execution.
AI-written summary · about
chapters (13)
- 0:00Intro
- 0:53Sebastiaan Zeeff
- 3:53From source code to execution
- 4:26Adding a new binary operator: A pipe operator
- 7:19Tokenization: from raw text to a stream of tokens
- 8:36Tokenization: Add a token for the operator
- 11:06Python's Grammar & the PEG Parser
- 14:30Implementing our new grammar rule
- 19:20Regenerate the parser based on the new grammar
- 20:17Part 2: From Abstract Syntax Tree to Magic (Execution) Part 2
- 22:34Making the compiler use the new 'opcode File: Python/compilec
- 25:09Part 2: We've got Bytecode!
- 29:35Recap & Remarks
related talks
28mPipe-ifying Python: The how and why of interpreter hacking
27mCPython's Compilation Pipeline
02h 26mPointers in Python
31mUnderstanding Python from the inside: C concepts to dive into CPython fearless
22mLow Level Python a Deep dive into the python sources
27mOh You Who Enter, Abandon All Hope
36mHacking the Python AST
24mThe AST and Me