Enabling VLLM V1 on AMD GPUs With Triton
summary
vLLM V1 runs on AMD MI300-series GPUs by replacing legacy HIP primitives with Triton kernels and by splitting heterogeneous batches into specialized kernels for prefill and decode. Kernel changes include new launch grids and tiling, reducing warps to avoid register spilling, aligned loads to a single KV block, loop unrolling, and forcing matrix-core-friendly layouts. The combined optimizations produce about a 4.7x speedup over the Triton baseline and expose paged KV cache and prefix-caching features with comparable or better time‑to‑first‑token than V0.
AI-written summary · about
chapters (14)
- 0:00VLLM V1 on AMD GPUs
- 1:28Need for new attention backend
- 3:35Triton for AMD GPU support
- 4:59Attention kernel terms
- 6:25Baseline kernel issues
- 8:32Optimizing decode sequences
- 10:38Grouped query attention handling
- 11:43AMD parallel optimizations
- 14:09Memory efficiency refactor
- 15:34Vectorized memory access
- 17:20Decode kernels V0 to V1
- 19:26Unified attention kernel
- 20:36Benchmarks V0 vs V1 on MI300X
- 21:40Future benefits of V1
related talks
24mPortable High‑Performance LLM Serving: A Triton Backend for...
31mHow IBM Research Achieved vLLM Platform Portability with Triton Autotuning
30mOptimizing vLLM for Intel CPUs and XPUs
34mBridging the Gap: Compiling and Optimizing Triton Kernels Onto RISC-V Targets Based on MLIR
08mGPU Programming with Triton Kernels
31mThe Evolution of Multi-GPU Inference in vLLM
26mVLLM-Helion: SOTA LLM Performance by Advanced Autotuning and Fine-grained Dispatching
10mMaximizing Kernel Development Productivity Under Performance Constraints