ATP: In-network Aggregation for Multi-tenant Learning

on YouTube ↗12m

" summary

Claims up to 38–66% training throughput improvement for multi-tenant distributed DNN jobs by moving gradient aggregation into programmable switches. Implements in-network aggregation in P4 on Barefoot Tofino-style hardware, using dynamic per-packet aggregators hashed by gradient index, two-level TOR+aggregation-switch reduction, and a best-effort fallback that forwards missed aggregations to parameter servers. Ensures exactness via sequence-detect retransmit, converts floats to 32-bit scaled integers to avoid switch FP, and evaluates on a cluster against static partitioning and prior systems.

AI-written summary · about

" chapters (10)

1. 0:00Introduction
2. 0:28Parameter Server Architecture
3. 3:06Agenda
4. 3:31Aggregation
5. 4:41Best Effort
6. 7:45Additional Challenges
7. 9:46Evaluation
8. 10:19Dynamic Allocation
9. 10:46Results
10. 11:39Summary

" related talks

• 16mAccelerating Distributed Graph Learning by Using Collaborative In-Network...Kai Chen, Junxue Zhang, Zhaoyi Li, Jiawei Huang, Yijun Li, Jingling Liu, Hui Li, Xiaojun Zhu, Shengwen Zhou, Jing Shao, Xiaojuan Lu, Qichen Su, Jianxin Wang, Chee Wei Tan, Yong Cui · USENIX ATC '25
• 39mScalable Multi-Node AI Workloads in Multi-Tenant AI Clouds U...Girish Moodalbail, Leonid Grossman · CloudOpen at Open Source Summit Europe 2024
• 14mBrain-on-Switch: Towards Advanced Intelligent Network Data Plane via NN-Driven Traffic...Qi Li, Jinzhu Yan, Haotian Xu, Zhuotao Liu, Ke Xu, Mingwei Xu, Jianping Wu · 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI '24)
• 19mTowards Domain-Specific Network Transport for Distributed DNN TrainingJunchen Jiang, Kai Chen, Hao Wang, Yong Wang, Han Tian, Xinchen Wan, Jingrong Chen, Wei Bai, Jiacheng Xia, Gaoxiong Zeng · 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI '24)
• 17mZEN: Empowering Distributed Training with Sparsity-driven Data SynchronizationYuke Wang, T. S. Eugene Ng, Zhuang Wang, Zhaozhuo Xu, Jingyi Xi, Anshumali Shrivastava · OSDI '25
• 17mTraffic Engineering for AI Training NetworksShuqiang Zhang, Jingyi Yang · Networking @Scale 2023
• 17mNetHint: White-Box Networking for Multi-Tenant Data CentersWei Zhang, Ion Stoica, Jingrong Chen, Danyang Zhuo, Hong Zhang, Liang Luo, Jeffrey Chase · 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI '22)
• 14mMultitenancy for Fast and Programmable Networks in the CloudXin Jin, Aurojit Panda, Hang Zhu, Tao Wang, Dan R. K. Ports, Anirudh Sivaraman, Fabian Ruffy · HotCloud '20