Themis: Fair and Efficient GPU Cluster Scheduling
summary
Proposes a new finish-time fairness (Rho) metric and a two-step allocator: filter top (1−f) apps then run a partial-allocation auction that computes a proportional-fair allocation with hidden payments to enforce truthfulness. Uses per-job progress pulled from hyperparameter optimizers via a Themis agent and is implemented on Hadoop 3.2 + Submarine on a 20‑machine 64‑GPU testbed with a Microsoft production trace. Claims max Rho ≈1.2, improves fairness >2.25× and cluster efficiency by ~5%–250%, and finds a fairness knob f≈0.8 balances incentive and packing.
AI-written summary · about
chapters (18)
- 0:00Intro
- 0:22Deep Learning at a Large Enterprise
- 1:38GPU Cluster Scheduler: Goal
- 2:49Existing GPU Cluster Schedulers
- 4:43GPU Cluster Scheduler: Drawback 2
- 6:51GPU Cluster Scheduler: Requirement 2
- 7:06Towards a new GPU Cluster Scheduler
- 7:21Themis: Metric
- 8:00Themis: Finish-Time Fairness Metric
- 8:38Themis: Interface
- 11:30Strawman Mechanism: Issues
- 13:09Themis: Mechanism: Partial Allocation Auction
- 14:46Themis: Overall Design
- 15:10Themis: Implementation
- 15:31Themis: Evaluation
- 15:48Macrobenchmark: Sharing Incentive
- 16:29Macrobenchmark: Efficiency
- 18:23Conclusion
related talks
12mElastic Resource Sharing for Distributed Deep Learning
24mTiresias: A GPU Cluster Manager for Distributed Deep Learning
28mMultitenancy and Fairness at Scale with Kueue: A Case Study
17mTessera: A Holistic Pipeline Parallelism Framework for Trillion-Parameter Heterogeneous
14mPollux: Co-adaptive Cluster Scheduling for Goodput-Optimized Deep Learning
20mScheduler and Sharding Considerations for Network Efficiency
10mFair Scheduling for Deep Learning Workloads in Kubernetes
20mBoosting Training and Inference Performance via Topology-Aware Scheduling of Heterogeneous Resources