Lightning Talk: Improved GEMM and SDPA Performance on ROCm With Composable Kernel
summary
Composable Kernel (CK) is now a ROCm backend for GEMMs and SDPA, reachable by build-time env vars like use_rocm_ck_gemm and a single Python line to select the backend; tuning uses the CK profiler. On MI300X BF16 workloads CK shows dramatic wins—GEMMs report an average 78.9% avg speedup for tuned pipelines, SDPA forward around 19% faster and backward nearly 98% faster versus incumbent kernels—while untuned shapes may still favor HiPLASs/other backends.
AI-written summary · about
chapters (9)
- 0:00Composable Kernel intro
- 1:27What is Composable Kernel
- 2:53CK for GEMM in PyTorch
- 3:57GEMM performance gains
- 5:45CK Profiler for tuning
- 6:48CK for SDPA integration
- 8:13SDPA performance boost
- 9:16CK across frameworks
- 9:58Future improvements
related talks
08mLightning Talk: Achieving SOTA GEMM Performance: A CuTeDSL Backend for PyTorch Induc...
12mLightning Talk: PyTorch 2.0 on the ROCm Platform
09mUsing and fine-tuning Gemma 3
20mBringing PyTorch Monarch to AMD GPUs: Single-Controller Distributed Tra...
32mPowering Your Generative AI Workloads with AMD and Open-Source ROCm
12mLightning Talk: Accelerating Inference on CPU with Torch.Compile
16mGMI-DRL: Empowering Multi-GPU DRL with Adaptive-Grained Parallelism
40mHidden GEMMs: How Optimized Math Libraries Work