Lightning Talk: Scaling Distributed Deep Learning with Service Discovery - Yong Tang
summary
Distributed TensorFlow clusters on AWS use CoreDNS plugins for service discovery so cluster membership and hostnames exist before nodes start, enabling one‑command launches and resilience to partial node failures. Implementations include a CoreDNS plugins approach that reads the EC2 instance launch index from instance metadata to provide persistent hostnames, and an alternative using an etcd backend that avoids persistent storage but requires running etcd. Deployments report simplified researcher workflows and stable training runs.
AI-written summary · about
chapters (5)
- 0:00Start
- 0:19Machine Learning @ Infoblox
- 0:58Machine Learning Infrastructure
- 2:13CoreDNS and Plugins
- 2:48Service Discovery with CoreDNS
related talks
43mDeep learning beyond the learning
32mDeep Learning with TensorFlow, PyTorch and MXNet on AWS
12mLightning Talk: Benchmarking Your Distributed ML Training on the K8s Platform
41mScaleable Distributed Deep Learning
04mProject Lightning Talk: Empowering Federated Learning with Multi-Cluster Management for Privacy and Efficiency
49mDistributed Deep Learning
34mDemocratizing Machine Learning on Kubernetes [I]
14mLightning Talk: Large-Scale Distributed Training with Dynamo and...