Fair Scheduling for Deep Learning Workloads in Kubernetes
summary
Kubernetes v1.15 scheduler framework is rejected in favor of a custom scheduler that uses a two-dimensional heap where each user is a heap node and each user node contains per-user pending heaps sorted by priority and creation time. Allocation pops the least-served user then that user’s next pod so GPUs alternate between users rather than by pod age. Long-running jobs (days or weeks) still get utilization but a preemption simulation periodically steals excess GPUs (example: 4 GPUs → 2 each) and reassigns pods to enforce fairness.
AI-written summary · about
chapters (6)
related talks
35mSIG Scheduling Deep Dive
31mSIG Scheduling Update: Transition From Pod To Workload Scheduling
28mMultitenancy and Fairness at Scale with Kueue: A Case Study
42mDeep Dive Into the Latest Kubernetes Scheduler Features
30mDeep Dive: Scheduling SIG
36mCapacity Scheduling for Elastic Resource Sharing in Kubernetes
28mAI Workload Preemption in a Multi-Cluster Scheduling System at Bloomberg - Leon Zhou & Wei-Cheng Lai
36mDeep Dive: SIG Scheduling