Voltrix: Sparse Matrix-Matrix Multiplication on Tensor Cores with Asynchronous...
summary
Voltrix-SpMM implements a CUDA 12.6 kernel for H100 that combines a bitwise-compressed BMAC format (128-bit blocks loaded with LDGXT) with TMA-driven asynchronous loads and a warp-specialized producer-consumer pipeline to hide memory latency, plus a persistent and I/O co-balanced kernel that partitions work to avoid atomic merges. The prototype on Hopper uses WGMM/TMA and reports average speedups of 36.5x vs TC-GNN, 1.8x vs DTC-SpMM and 1.7x vs RoDe, and the code is open source.
AI-written summary · about
chapters (11)
- 0:00Sparse matrix multiplication on Tensor Cores
- 1:47Tensor Cores and Hopper overview
- 3:36Sparse matrix multiplication challenges
- 5:24Existing solutions and limits
- 8:34Voltrix SPMM innovations
- 9:40Bitwise compression design
- 12:09Asynchronous pipeline model
- 13:55Persistent kernel balancing
- 15:21Implementation details
- 16:07Experimental setup
- 17:57Performance results
related talks
08mGPU Programming with Triton Kernels
17mA Sparse Iteration Space Transformation Framework for Sparse Tensor Algebra
10mMaximizing Kernel Development Productivity Under Performance Constraints
01h 18mOptimizing LLM Training and Inference Performance on GPUs
22mEnabling VLLM V1 on AMD GPUs With Triton
18mCompiling Strassen-like Matrix Multiplication Algorithms to Fast CUDA Kernels
24mPortable High‑Performance LLM Serving: A Triton Backend for...
17mSparseZETA: Intelligent Auto-tuner for Designing High-Performance SpMV Programs