Pync: Function-Level Incremental Execution for Python Scripts
summary
Pync incrementally executes Python scripts at the function-level incremental execution granularity by source-to-source instrumenting code to emit an event-log instrumentation of loads, writes and call graphs, then analyzes those logs to detect pure calls and store results in a persistent cache. The prototype restores cached function results instead of re-running expensive calls, claims 1.6× and 4.8× re-execution speedups on two data-science benchmarks, and reports modest first-run overheads (about 1.2× and 1.04×) while correctly avoiding caching impure writes.
AI-written summary · about
chapters (11)
- 0:00Pync for incremental Python
- 1:25Notebooks unreliable for increment
- 2:49Detecting side effects in Python
- 3:54Tracking writes and caching
- 5:16Preliminary evaluation and gains
- 6:20Conclusion and future
- 6:42Q&A: File system interactions
- 7:25Q&A: Extending beyond wrappers
- 7:47Q&A: Thresholds for expensive
- 8:28Q&A: Dynamic features and overload
- 9:12Q&A: Input-dependent dependencies
related talks
22mPyPy hands-on
24mRestarting Pyjion, a general purpose JIT for Python- is it worth it?
22mPyPy and the future of the Python ecosystem
22mPyPy in production
22mPyPy hands-on
33mA low latency deepdive of Python with Cython
26mPython Performance at Scale - Making Python Faster at Instagram
48mPython Performance: Past, Present and Future