Exploiting Sparsity and Redundancy
summary
Builds on Apache SystemDS with a DSL and compiler that auto-switches sparse formats at runtime and adds GPU and federated backends, then generates fused C/Java operators to iterate nonzeros and skip dense intermediates, yielding up to orders of magnitude speedups on sparse matrix chains and a reported ~3x compression gain from co-coding correlated columns. Demonstrations include matrix factorization and cross-entropy losses, a cost-based fusion optimizer, the MNC sparsity sketch for planning, and compressed linear algebra using DDC and lightweight encodings to reduce memory bandwidth.
AI-written summary · about
chapters (8)
- 0:00Data-centric ML pipelines
- 1:59ML system primitives
- 5:39Sparsity in linear algebra
- 9:49Fused operator performance
- 11:15Sparsity benefits comparison
- 13:04Redundancy via compression
- 15:30Workload-aware compression
- 16:39Q&A
related talks
28mFunctional Sparse Tensor Compilation
20mA Mechanized Algebra of Verified Data Structures for Optimizing Sparse Tensor Programs
15mCompilation of Sparse Array Programming Models
30mGenerating High Performance Code for Irregular Data Structures using Dependent Types
19mCompiler Support for Sparse Tensor Convolutions
14mCompressed and Parallelized Structured Tensor Algebra
19mUniSparse: An Intermediate Language for General Sparse Format Customization
20mVectorizing Sparse Coiteration for Two-finger Loop Structure (Extended Abstract)