Optimizations and abstractions for sparse machine learning
summary
Holistic optimizations trade between sparse and dense work using matrix reassociations and an offline/online compiler called sensei that picks the fastest reassociation per graph and hardware, yielding up to 6× epoch speedups on TGNNs. Temporal redundancy eliminations (duplication, memorization, precomputation) power a TGlite-style framework for reuse across models. For moderate-density attention (20–50% nonzeros) a new Fast Compressed Sparse Row format cuts metadata from O(nnz) to O(rows) and a GPU code generator splat consistently beats cuBLAS/Kuspa/Triton kernels in that regime.
AI-written summary · about
chapters (6)
- 0:00Introduction
- 1:45Sparse relations in ML
- 3:51Sparse vs dense in GNNs
- 7:49Temporal GNN optimizations
- 13:46Moderate sparsity in attention
- 18:02Summary
related talks
18mTC-GNN: Bridging Sparse GNN Computation and Dense Tensor Cores on GPUs
40mSparse models are fast models: Improving DNN inference performance by over 10X
37mUnstructured Sparsity Meets Tensor Cores: Lessons from Sparse Attention and MoE
28mFunctional Sparse Tensor Compilation
19mGeneralSparse: Bridging the Gap in SpMM for Pruned Large Language Model Inference..
17mZEN: Empowering Distributed Training with Sparsity-driven Data Synchronization
20mVectorizing Sparse Coiteration for Two-finger Loop Structure (Extended Abstract)
19mCompiler Support for Sparse Tensor Convolutions