Sparse Computing Drives Energy-Efficient Computation for Artificial Intelligence
summary
Presents a GPU-focused sparse-kernel library dgSPARSE with an auto-selection model that outperforms commercial kernels across GNN models and sparse operators, shows FPGA inference system FlightLLM achieving 6.0x energy efficiency versus V100S-class GPUs, and describes a spatial-temporal sparsification plus mixed-precision design on an AMD V80 FPGA that yields a 1.3x end-to-end speedup for video generation despite the GPU having ~21x peak compute.
AI-written summary · about
chapters (8)
- 0:00Sparse computing for AI
- 2:45AI demand vs chip performance
- 6:21Sparse computing in AI 1.0
- 9:58Dynamic API for sparse matrices
- 13:32Foundation models challenges
- 16:22Structured sparsity in LLMs
- 17:47Video generation with sparsity
- 21:22Summary and future
related talks
40mSparse models are fast models: Improving DNN inference performance by over 10X
01h 52mNeural Network Accelerator Design
13mHardware/Software Co-Programmable Framework for Computational SSDs to Accelerate Deep...
19mGeneralSparse: Bridging the Gap in SpMM for Pruned Large Language Model Inference..
01h 11mMachine Learning Acceleration in an Embedded World
18mTC-GNN: Bridging Sparse GNN Computation and Dense Tensor Cores on GPUs
17mZEN: Empowering Distributed Training with Sparsity-driven Data Synchronization
16mTowards High-throughput and Low-latency Billion-scale Vector Search via CPU/GPU...