Bagua: Lightweight Distributed Learning on Kubernetes
Talkat KubeCon + CloudNativeCon + Open Source Summit China 2021 · on Thursday, 2021-12-09· at 14:05· Kubecon + CloudNativeCon 演讲厅, 虚拟会议 (Virtual)·
summary
Bagua is a distributed deep-learning system that runs on Kubernetes using ordinary Ethernet and without special network hardware or strict scheduling. It implements novel communication algorithms and Kubernetes integrations to horizontally scale training for models like ResNet and BERT and for very large industrial workloads (dozens of TB of parameters, billions of images/videos, TB-scale ASR datasets). In production Kubernetes clusters Bagua reports end-to-end training speedups versus PyTorch-DDP, Horovod, and BytePS (up to ~1.95×).
AI-written summary · about
chapters (7)
- 0:00Start
- 5:48Comprehensive Support on K8S
- 6:53Training Task Auto-Scaling on K8S
- 7:48Scheduling Policy and Cascading Resource Queue
- 9:38Cluster Controller for Kas Security & Performance
- 10:52Bagua-Operator for KSS Security & Performance
- 11:54Bagua-Operator Usage Example
related talks
34mDemocratizing Machine Learning on Kubernetes [I]
22mYi Wang, Tech Lead of AI Platform, Baidu, NYC 2017
34mDeep Customized Kubernetes for Machine Learning in Tencent
31mKubernetes For GPU Powered Machine Learning Workloads In Academia
32mKoba: Disaggregated Data Loading for Multimodal Training at Luma AI
13mHow to scale Neural Network Architecture Search with RabbitMQ and Kubernetes
12mLightning Talk: Benchmarking Your Distributed ML Training on the K8s Platform
25mScaling Machine Learning Models on Kubernetes