Lightning Talk: Distributed AI Without the Infrastructure Tax
summary
Provides an actionable stack: open-source Neuron Deep Learning Containers that lock PyTorch, Neuron SDK, XLA and PrivateUse1 as a tested contract to avoid dependency hell; a Kubernetes Dynamic Resource Allocation driver that slices Neuron cores, offers topology-aware scheduling and multi-tenant device claims so manifests scale without node selectors; and preconfigured EFA zero-copy settings to avoid cross-NUMA slowdowns that can cost 10x throughput. Shows same image scaling from laptop to 32-node and claims 65–80% cluster utilization with a GitHub repo.
AI-written summary · about
chapters (8)
- 0:00Distributed AI challenges
- 1:27Minimizing infrastructure tax
- 3:34Problem breakdown and dependencies
- 6:01Three-layer solution
- 7:51Deep learning containers
- 8:36Abstracting hardware with DRA
- 10:22Network for tensor transfer
- 11:48Summary
related talks
23mSponsored Session: Powering PyTorch: Decentralized Training for an Energy-Hungry Future
16m[Opening Keynote] Making Distributed Computing Easy
43mKeynote: Building AI in the Open: How the Cloud Native Community Shapes What Comes Next
05m⚡ Lightning Talk: Using Decentralized File System to Optimize ML/AI Workloads on Kubernetes
22mAI @Scale
38mMPI Meets Machine Learning: Unlocking PyTorch distributed for scaling AI workloads
49mDistributed Deep Learning
04mA Cloud Native Stack From Bare Metal To Tokens for Large-Scale AI Inference