Efficient Multi-Cluster GPU Workload Management with Karmada and Volcano
Talkat Cloud Native AI Day Europe 2024 · on Tuesday, 2024-03-19· at 14:05· Pavilion 7 | Level 7.3 | Room S03·
summary
Karmada and Volcano combine to run gang-scheduled GPU jobs across clusters using Prometheus metrics, a scheduler estimator that simulates in-cluster scheduling, and a resource model; the estimator finds clusters able to run all replicas (example: cluster A can run 10 replicas while B/C only partial) and enforces rebalancing and failover via cluster groups and status layers. GPU sharing uses CUDA-level isolation. The control plane propagates CRDs to member clusters. Karmada Volcano
AI-written summary · about
chapters (9)
- 0:00Multi-cluster GPU workloads
- 3:59Ideal GPU platform capabilities
- 6:11Karmada and Volcano architecture
- 8:20AI workload job abstraction
- 11:14Gang scheduling across clusters
- 13:24Karmada resource management
- 17:07Multi-layer scheduling models
- 20:45GPU virtualization challenges
- 22:11Q&A
related talks
27mBalancing Cost and Efficiency: Day2 Optimization of Multi-Cluster AI Infrastructure
26mIntro: Kubernetes SIG
24mCluster Management for Large Scale AI and GPUs: Challenges and Opportunities
28mEfficient Scheduling Of High Performance Batch Computing For Analytics Workloads With Volcano
32mVolcano: Intro & Deep Dive
22mKeep HPC Running - an SRE's Guide to Supporting GPUs on Kubernetes
31mShared GPU Scheduling & Proactive Autoscaling: A Production Blueprint for 1000+…
08mNext-Gen AI Orchestration With Volcano On Kubernetes