Optimizing Dynamic Neural Networks with Brainstorm
summary
Brainstorm introduces Cell as the sub-tensor routing unit and Router as a programmable dispatch operator, collects fine-grained runtime profiles, and applies three concrete optimizations: dynamic horizontal fusion to fuse imbalanced branch kernels, placement that co-locates highly correlated expert paths to cut inter-GPU communication, and speculative routing that skips routers when activations are biased. Evaluations on MoE, Switch Transformer, Task MoE, SR and MTNet report up to 11.7× speedup (3.29× average) and up to 42% memory reduction.
AI-written summary · about
chapters (10)
- 0:00Optimizing dynamic neural networks
- 1:10Dynamic networks in practice
- 2:36Challenges in optimization
- 3:39Brainstorm’s goals and abstractions
- 5:07Defining dynamic models
- 6:51Brainstorm’s system architecture
- 7:56Dynamic horizontal fusion
- 9:21Correlated path placement for MoE
- 10:47Speculative routing acceleration
- 12:13Summary and future research
related talks
14mBrain-on-Switch: Towards Advanced Intelligent Network Data Plane via NN-Driven Traffic...
25mDaydream: Accurately Estimating the Efficacy of Optimizations for DNN Training
01h 02mMulti-Dimension Mapping of Embedded Applications
22mOptimizing Large MoE Inference on NVIDIA Blackwell: NVFP4, ADP, and DualPipe Strat...
51mTips & Tricks for Fast Neural Net Inference in Production
19mOffloading-Efficient Sparse AI Systems
43mKeeping Moore's law alive: Neuromorphic computing
40mOptimize Your AI Cloud Infrastructure: A Hardware Perspective