A Sparse Iteration Space Transformation Framework for Sparse Tensor Algebra
summary
Framework implements sparse iteration-space transformations inside TACO and a CUDA backend and generates fused sparse kernels (SpMV, SpMM, SDDMM, MTTKRP) that match or beat hand-tuned code across SuiteSparse: 6.9x GPU speedup versus naive TACO GPU parallelization, 36% extra from precompute+unroll, and CPU results of 3.34x over TACO without transforms and 1.49x over SPLATT. It includes an autoscheduler, produces load-balanced warp/thread mappings, and reports geometric-mean speedups against cuSPARSE on irregular matrices.
AI-written summary · about
chapters (9)
- 0:00Many important problems are messy and irregular
- 0:42Too many tensor kernels for a fixed-function library
- 1:04Fused kernels are asymptotically faster
- 1:48Sparse transformations are necessary
- 3:55Sparse transformations are challenging
- 5:23A Sparse Transformation Framework
- 6:40Separation of algorithm, data representation, and schedule
- 9:36Performance compared to hand-optimized
- 12:44Load-balancing sparse computations
related talks
15mCompilation of Sparse Array Programming Models
20mCompiling Structured Tensor Algebra
28mFunctional Sparse Tensor Compilation
18mCompilation of Shape Operators on Sparse Arrays
28mC2TACO: Lifting Tensor Code to TACO
19mCompiler Support for Sparse Tensor Convolutions
20mSparseAuto: An Auto-Scheduler for Sparse Tensor Computations Using Recursive Loop Nest(…)
25mFormat Abstraction for Sparse Tensor Algebra Compilers