Insum: Sparse GPU Kernels Simplified and Optimized with Indirect Einsums
summary
Introduces Insum, a compiler that encodes sparse formats as indirect Einsums and lowers them into an extended PyTorch/Torch-Compiler pipeline to generate Tensor‑Core–aware GPU code; implements two fixed-length formats GroupCOO and BlockGroupCOO and a fused code path that replaces PyTorch’s fallback gather/compute/scatter. Evaluation on an RTX 3090 shows 0.94×–3.81× speedups across benchmarks, up to ~4× over handwritten kernels in some formats, and 92×–4491× fewer lines of code.
AI-written summary · about
chapters (6)
- 0:00Introduction
- 4:44Sparsity in kernels
- 7:14Fixed vs variable formats
- 9:44Indirect Einsums and PyTorch
- 11:10Performance results
- 13:20Impact on ML workflows
related talks
08mLightning Talk: Achieving SOTA GEMM Performance: A CuTeDSL Backend for PyTorch Induc...
18mTC-GNN: Bridging Sparse GNN Computation and Dense Tensor Cores on GPUs
37mUnstructured Sparsity Meets Tensor Cores: Lessons from Sparse Attention and MoE
10mCombo Kernels: Horizontal Fusion Optimizat...
27mAccelerating Generative AI
23mOffload Annotations: Bringing Heterogeneous Computing to Existing Libraries...
30mHighLoad Python SIMD, GPU, and Horizontal Scaling Looking into Silicon
01h 04mCompile-time sparse matrices