Mirage: A Multi-Level Superoptimizer for Tensor Programs
summary
Mirage is a multi-level superoptimizer that generates custom CUDA kernels from tensor programs using a uniform representation across kernel, thread-block, and thread levels. It enumerates candidate µGraphs, prunes the search with abstraction-guided pruning and an automated theorem prover, and checks correctness via a probabilistic equivalence verification with theoretical guarantees. Evaluation shows up to 3.3× speedup over prior systems and a discovered single-kernel H100 micrograph that is 2× faster than the best existing kernel; source is on GitHub.
AI-written summary · about
chapters (9)
- 0:00Mirage intro
- 1:43GPU kernel limits
- 3:05Multi-level optimization
- 5:13MuGraph IR design
- 7:41Exhaustive search & verification
- 8:46Pruning techniques
- 13:01Benchmark results
- 13:44Case study group query attention
- 14:46Conclusion
related talks
15mOptimus: Accelerating Large-Scale Multi-Modal LLM Training by Bubble Exploitation
15mGraCE: Unlocking CUDA Graphs with Compiler Support for ML Workloads
24mVerified Tensor-Program Optimization Via High-Level Scheduling Rewrites
42mAnima Anadkumar, Principal Scientist, Amazon Web Services, Endowed Professor, CalTech
18mCompiling Strassen-like Matrix Multiplication Algorithms to Fast CUDA Kernels
21mKTransformers: Unleashing the Full Potential of CPU/GPU Hybrid Inference for M...
11mAccelerated Software for a Post-Moore World
32mPanel Discussion: The AI Kernel...