Challenges with Hardware-Software Co-design for Sparse Machine Learning on (...) Dataflow
summary
Extends the Sparse Abstract Machine (SAM) and a dataflow accelerator model to handle real ML models, adds new hardware primitives (filter block, unary-value stream, memory movement) and a DeepL compiler to handle kernel fusion and iteration-order selection. Shows that fusing attention kernels yields more than 2x performance on their benchmarks, finds three concrete pain points—missing nonlinear/data-movement ops, compiler-driven dataflow ordering, and varied sparsity formats—and demonstrates prototype solutions that improve end-to-end sparse ML execution.
AI-written summary · about
chapters (6)
- 0:00Hardware-software co-design challenges
- 1:25Generality in dataflow accelerators
- 3:09Sparse ML dataflow accelerator
- 5:15Optimizing dataflow for sparse models
- 8:06Sparse data patterns representation
- 10:12Conclusion and future directions
related talks
23mSparse Computing Drives Energy-Efficient Computation for Artificial Intelligence
01h 53mMachine Learning for Manycore System Design and Optimization
19mOptimizations and abstractions for sparse machine learning
20mVectorizing Sparse Coiteration for Two-finger Loop Structure (Extended Abstract)
54mProgramming Language Design and Implementation for the Machine Learning Era: A Personal(…)
37mUnstructured Sparsity Meets Tensor Cores: Lessons from Sparse Attention and MoE
34mOptimizing AI Inference with ML Compilers & Hardware
28mFunctional Sparse Tensor Compilation