Unlocking Performance: Harnessing LLMs To Streamline GPU Kernel Development in...
summary
KernelBook provides 25,000+ PyTorch→Triton pairs as training data and KernelLLM yields surprising efficiency: an 8B-parameter model matches 671B DeepSeek‑R1 on pass@10, with an 80B model in training for translation tasks. BackendBench validates kernels with OPInfo and real Hugging Face tensor shapes, publishes 84 example kernels, finds most LM kernels run at ~70–100% of PyTorch baseline with a few up to +20%, and shows LM kernels work for forward passes but still fall back for reliable backward implementations.
AI-written summary · about
chapters (10)
- 0:00GPU kernel development intro
- 1:05Kernel development trade-offs
- 2:30Performance gains from kernels
- 3:56Benchmarking LM-driven authoring
- 6:45Correctness in kernel generation
- 8:11GPU kernel benchmarking pitfalls
- 11:22BackendBench framework
- 12:46LM kernel performance results
- 13:29Limitations and future work
- 14:11Q&A
related talks
24mPortable High‑Performance LLM Serving: A Triton Backend for...
26mVLLM-Helion: SOTA LLM Performance by Advanced Autotuning and Fine-grained Dispatching
24mOptimizing CPU LLM Inference in PyTorch: Lessons From VLLM
08mGPU Programming with Triton Kernels
31mHow IBM Research Achieved vLLM Platform Portability with Triton Autotuning
10mMaximizing Kernel Development Productivity Under Performance Constraints
08mHacks to Make LLM Training Faster
16mWLB-LLM: Workload-Balanced 4D Parallelism for Large Language Model Training