Intro: Kubernetes SIG
summary
Kubernetes lacks batching for HPC/ML workloads so Volcano provides a CRD-based job abstraction, a volcano scheduler and controllers that implement gang scheduling (all-or-nothing starts), namespace fair-share by proportion and optional backfill policies. Tests show a six-pod distributed training job finishes in ~24 minutes under Volcano versus some runs stuck pending for >20 hours under the default scheduler. Current release adds job management; roadmap lists queue/hierarchical queues, preemption/reclaim, GPU/FPGA and topology-aware scheduling.
AI-written summary · about
chapters (12)
- 0:00Introduction to speakers
- 0:43Batch workload challenges
- 4:05Job scheduling solutions
- 5:03Job starvation and backfill
- 7:02Backfilling policies
- 9:20Namespace resource sharing
- 11:59Batch workload requirements
- 13:56Additional required features
- 14:46Volcano project overview
- 17:38Volcano components
- 19:40Roadmap and resources
- 21:19Q&A session
related talks
32mVolcano: Intro & Deep Dive
24mIntro To Volcano: Cloud Native Batch System
43mRun Your AI Workloads and Microservices on Kubernetes More Easily and Efficiently...
24mVolcano – Cloud Native Batch System for AI, BigData and HPC
28mEfficient Scheduling Of High Performance Batch Computing For Analytics Workloads With Volcano
37mImproving Performance of Deep Learning Workloads With Volcano
30mVolcano: A Unified Scheduling Platform for Cloud Native AI
08mNext-Gen AI Orchestration With Volcano On Kubernetes