More Nodes, More Problems: Solving Multi-Host GPU/TPU Scheduling With Dynamic Resource Allocation
Talkat KubeCon + CloudNativeCon Europe 2025 · on Wednesday, 2025-04-02· at 14:30· Level 1 | Hall Entrance S10 | Room A·
summary
DRA beta in Kubernetes 1.32 enables modeling and claiming devices that span nodes, so a 16‑node cluster with 4 TPUs each (64 TPUs total) can advertise overlapping logical slices and satisfy a single resource claim for 16 TPUs (four nodes) referenced by four pods. The current allocator uses first‑fit scheduling; 1.33 adds partitionable devices in 1.33 (alpha) and device taints trigger automatic eviction so a failed node causes all pods on a claimed multi‑host device to be evicted and rescheduled to a new slice.
AI-written summary · about
chapters (9)
- 0:00Kubernetes multi-host GPU/TPU scheduling
- 1:24Large multi-node accelerator workloads
- 8:11Dynamic Resource Allocation (DRA) solution
- 13:34DRA models for multi-host TPUs
- 17:28Scheduling pods with DRA claims
- 19:19Handling failures with DRA taints
- 20:25Open issues in DRA
- 21:57DRA for GPU and TPU workloads
- 23:06Q&A
related talks
42mGuide Us on Next Steps in Kubernetes Device Manage...
29mWhat Can I Get You? An Introduction to Dynamic Resource Allocation
50mDynamic Resource Allocation - the path towards GA
21mSustainability by Design: Leveraging DRA for Energy-Efficient…
34mUnlocking Performance: Topology-Aware CPU Scheduling W...
34mKubernetes DRA Architecture: Scheduling, Status, and Topology at Scale
24mFit-to-Serve: How a New DRA Capability for Dynamic Device...
32mScaling AI Workloads with Kubernetes: Sharing GPU Resources Across Multiple Containers