Practical Python parsing with the ast standard module
summary
Shows how to use the standard ast library to parse Python code with ast.parse and inspect nodes via ast.walk and NodeVisitor/NodeTransformer. Demonstrates a live run over the pydantic codebase to collect per-function metrics (line count, character count) and plots correlation, finds instances where dataclasses are applied via calls not names, and rewrites list comprehensions into set comprehensions. Also covers helpers like ast.get_source_segment, ast.get_docstring and ast.unparse for regenerating modified code.
AI-written summary · about
chapters (14)
- 0:00Introduction to ast
- 1:07Why use the ast module
- 3:11Getting the AST from code
- 4:56Understanding AST nodes
- 8:27Common AST node types
- 10:55Traversing the AST
- 14:26Node visitors and transformers
- 17:59Other AST helpers
- 18:39Demo overview
- 19:00Analyzing a codebase
- 20:45Visualizing function stats
- 21:25Checking for dataclasses
- 22:50When to use ast
- 23:32Q&A
related talks
17mUnlocking Python's AST: The Metaprogramming Superpower You Didn't Know You Had
27mDigging into Abstract Syntax Trees for analyzing and modifying Python programs
30mAdvanced parsing of structured data using Python's new match statement
27mAST Meta-programming
29mMy Little Parser: Tooling is Magic
46mPython ships a ready-made frontend for your compiler
27mOh You Who Enter, Abandon All Hope
44mJavaScript AST programming: Introduction and beyond