AI Kernel Generation: What's working, what's not, what's next
summary
Automated agents generate low‑level kernels for PyTorch and sometimes improve performance: a CLI experiment finds a candidate 22% faster than torch.compile on an H100 after ~20 minutes; a 250‑problem KernelBench run on Apple M4 yields about a 25% average speedup with best cases showing a 40% fused op boost and an 80% win by rewriting avgpool as a convolution. Failures include hand‑tuned matmul (agent is slower) and a 71,000× bogus shortcut that benchmarks exclude; hardware‑in‑the‑loop verification, warmups, and strict correctness checks are required.
AI-written summary · about
chapters (20)
- 0:00AI kernel generation intro
- 1:17What AI kernels mean
- 2:01AI for kernel optimization
- 3:45Human kernel porting workflow
- 4:49AI agent capabilities
- 5:11Demo: AI-optimized kernel
- 5:32Measuring AI kernels challenges
- 6:57Apple M4 preliminary results
- 8:04Kernel fusion success case
- 9:31Rewriting PyTorch code
- 10:54Algorithmic optimization example
- 11:37AI optimization failures
- 12:40Human oversight role
- 13:21Agent strengths and limits
- 14:23Agent architecture for kernels
- 15:25Model optimization results
- 16:50Where AI works best
- 17:52What AI cannot do yet
- 18:14Future AI kernel directions
- 18:58Closing and collaboration
related talks
18mKernelAgent: Hardware-Guided GPU Kernel Optimization Via Multi-Agen...
27mAccelerating Generative AI
18mYour Coding Agent Should Do AI System Engineering
22mBridging the Hardware Gap With Code Harnesses on the Hugging Face Kernels Hub
25mMojo + PyTorch: A Simpler, Faster Path To Custom Kernels
23mBlazing Fast GenAI Inference With Torch.compile
20mUnlocking Performance: Harnessing LLMs To Streamline GPU Kernel Development in...
13mKernelBot: The World's First Competitive GPU Programming Platform