Source code processing with Python
summary
Shows how Python finds and rewrites real C and Objective C code by combining a Python parser generator, a C search object that iterates code and returns hits for specific function symbols, and a template library that performs token-aware templates substitutions. Demonstrates a search script that locates all calls and definitions of a function, a replacer that inserts missing tokens to keep code syntactically correct, and reports that token-level transforms are slower than plain parsing but produce valid, low‑noise edits.
AI-written summary · about
chapters (10)
- 0:00Source code processing intro
- 2:00Why regular expressions fail for code parsing
- 5:00Tree-sitter for code analysis
- 9:00Custom languages with grammars
- 13:00Searching code with queries
- 16:30Template-based substitutions
- 20:00Token-level transformations
- 23:40Expression generation
- 28:45Handling multiplicities
- 29:56Tree-sitter performance
related talks
27mDigging into Abstract Syntax Trees for analyzing and modifying Python programs
25mHow to bring your C++ project to Python land
27mPython: From Source to Execution
36mHacking the Python AST
31mUnderstanding Python from the inside: C concepts to dive into CPython fearless
47mWriting Python Codemods for Fun and Profit
01h 21mExploring Python Code Objects
05mHow to build a Code Search Engine