Triangulating Python Performance Issues with SCALENE
summary
Scalene is a low‑overhead profiler that simultaneously measures CPU, memory, and GPU usage and distinguishes Python vs native time by using virtual‑time deltas to attribute delays to C code; it introduces threshold‑based sampling for memory (sampling on megabyte deltas) to avoid the 300× slowdown of tracer profilers. It reports a new metric, copy volume, pinpoints numpy-induced copies (example: removing np.array cuts runtime 30s→23s and memory ~3GB→1.6GB), helped Semantic Scholar cut costs 92%, and supports LLM-driven code suggestions via OpenAI.
AI-written summary · about
chapters (11)
- 0:00Introduction to Python
- 0:34The problem of bloat
- 2:06Python's inefficiency
- 3:06Performance challenges
- 4:28Introducing Scalene
- 5:55Analyzing program metrics
- 7:14Optimization examples
- 8:23Profiling overhead
- 9:40Native vs Python time
- 12:14Low-overhead sampling
- 13:33AI-powered optimization
related talks
26mScalene: A high-performance, high-precision CPU+GPU+memory profiler for Python
39mPython Performance Matters
45mPython for Speed, Scale, and Science
25mScale Smarter, Not Harder, with cuPyNumeric.
30mHow to Tune your Python Analysis Pipeline: A Profiler Guide
30mHighLoad Python SIMD, GPU, and Horizontal Scaling Looking into Silicon
31m"Optimizing Scientific Python with C++, CUDA, and Serverless Compute"
32mProfiling at the speed of light