Advancements in High-Performance AI/ML through PyTorch's Python Compiler
summary
PyTorch 2.0 adds a Python-based compiler that captures program graphs with Dynamo, lowers them via FX and the frame-evaluation API (PEP 523), and emits GPU code with Torch Inductor using Triton. Benchmarks use 80 real-world models; 93% run unchanged and the compiled workload averages a 43% speedup on A100 GPUs. The system supports torch.compile as a drop-in API, adds runtime guards and recompilation, and keeps most execution in Python while moving heavy kernels to optimized code.
AI-written summary · about
chapters (12)
- 0:00Start
- 2:00What makes PyTorch so popular?
- 6:11Eager-Mode is slow
- 8:42Opposing goals
- 9:28Understanding a deep learning compiler
- 11:45Trace
- 13:38TorchScript
- 14:51on our Compiler Journey
- 15:01Dynamo
- 19:132.0 Compiler Stack
- 22:27Resources
- 22:52Follow PyTorch!
related talks
29mKeynote
15mPyTorch 2.0: Unlocking the Power of Deep Learning with the Torch Compile API
36mDL Compiler Panel Discussion
14mLightning Talk: Accelerating PyTorch Models With Torch.compile's C++ Wrapper Mode
13mPyTorch 2.0
23mBlazing Fast GenAI Inference With Torch.compile
32mKeynote: Fireside Chat with Soumith Chintala, Co-Creator of PyTorch & Merve Noyan
12mLightning Talk: Accelerating Inference on CPU with Torch.Compile