GPU Hunter: Architecting Global GPU Availability With MultiKueue
summary
Global GPU shortage shows up as a concrete bug: kubequeue quota can admit workloads without real GPU nodes, so quotas of 4 with a 5-replica deploy end up with only 3 pods actually running and the rest stuck with “capacity not found.” The team builds a MultiKueue manager/worker deployment, adds admission checks and the best effort atomic scale up provisioning class, and patches the scheduler to dispatch only to workers with verifiable node capacity across regions (issue #8089), which fixes the stranded-workload failure.
AI-written summary · about
chapters (16)
- 0:00Welcome
- 1:05Speakers and agenda
- 1:48AI workloads on Kubernetes
- 3:15What is Q
- 5:03Resource flavors and queues
- 6:32Multi-tenancy with Q
- 8:58MultiKueue for GPU pooling
- 10:21MultiKueue architecture
- 12:49User perspective and dispatch
- 14:20Global GPU scarcity
- 15:47Over-provisioning limitations
- 16:32Dynamic global scaling
- 17:17Demo setup
- 18:42Live demo
- 22:41Key takeaways
- 23:22Q&A
related talks
28mGPU Reservations: Maximizing Utilization and Fairness Across Teams
43mThe GPU Utilization Problem: What’s Going Wrong and How to Solve It
38mImproving GPU Utilization using Kubernetes
22mBeyond Single-Cluster Limits: Scaling GPU Workloads Across Kubernetes With…
28mMultitenancy and Fairness at Scale with Kueue: A Case Study
22mKeep HPC Running - an SRE's Guide to Supporting GPUs on Kubernetes
31mShared GPU Scheduling & Proactive Autoscaling: A Production Blueprint for 1000+…
10mLightning Talk: K8s Issue #52757: Sharing GPUs Among Multiple Containers