Lightning Talk: Adding Backends for TorchInductor: Case Study with Intel GPU
summary
Two integration levels exist: ATen/PrimsIR and the lower Inductor loop IR, and a dynamic registration mechanism is implemented so a backend can register codegen for a device at runtime and reuse Inductor features like loop fusion and memory planning. The Intel GPU backend sits on top of Triton, needs Triton load/store alignment and tuned workspace counts, and delivers conservatively ~19% training and ~15% inference speedups while shrinking training outliers from 24% to ~4% and removing inference outliers.
AI-written summary · about
chapters (7)
- 0:00Introduction to Intel GPU
- 0:39Inductor integration steps
- 3:02Architecture and design
- 4:24Performance analysis
- 7:56Triton and optimization
- 10:15Future work and summary
- 11:43Q&A session
related talks
08mA Halide Backend for TorchInductor
07mPyTorch 2.0: TorchInductor
08mLightning Talk: Achieving SOTA GEMM Performance: A CuTeDSL Backend for PyTorch Induc...
28mGenerating State-of-the-Art GEMMs for Heterogeneous Hardware with...
25mRace to Superintelligence: Building Low-Friction AI Infrastructure for a Heterogen...
36mDL Compiler Panel Discussion
24mPortable High‑Performance LLM Serving: A Triton Backend for...
10mLightning Talk: Faster Than SOTA Kernels in Torch.compile With Subgrap...