Scaling Distributed Machine Learning with In-Network Aggregation
summary
Implements SwitchML: a co-designed system that pushes aggregation into a programmable switch dataplane to reduce model-update volume and offload the all-reduce step. Uses simple integer/float packing and end-host protocol changes with existing ML frameworks, runs on Barefoot-style switches, and supports failure recovery in the switch-assisted protocol. Evaluated on real benchmarks and clusters, showing up to 5.5× speedup in training time by cutting network traffic and latency.
AI-written summary · about
chapters (6)
- 0:00In-network aggregation for ML
- 2:50Distributed ML training challenges
- 4:39Architecture for scaling ML
- 6:53Demo: In-network aggregation
- 9:01Performance benchmarks
- 10:35Future directions & Q&A
related talks
16mAccelerating Distributed Graph Learning by Using Collaborative In-Network...
19mTowards Domain-Specific Network Transport for Distributed DNN Training
38mMPI Meets Machine Learning: Unlocking PyTorch distributed for scaling AI workloads
19mOn Modular Learning of Distributed Systems for Predicting End-to-End Latency
17mModel-Switching: Dealing with Fluctuating Workloads in Machine-Learning-as-a-Service
34mDecentralized Machine Learning
16mCredence: Augmenting Datacenter Switch Buffer Sharing with ML Predictions
48mFederated ML for Edge Applications