Parcae: Proactive, Liveput-Optimized DNN Training on Preemptible Instances
Talkat 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI '24) · 2024-04-16..04-18·
summary
Parcae is a system that proactively adapts hybrid data+pipeline parallelization to predicted spot-instance changes, using a liveput metric that balances throughput and robustness, an availability predictor that forecasts short‑term instance counts from time‑series traces, a dynamic‑programming optimizer that searches strategy sequences, and lightweight instance migration primitives to reconfigure without full checkpoints. Evaluations on multiple DNNs and AWS spot traces show up to 10× throughput improvement over Veruna/Bamboo and near‑optimal performance versus an oracle under frequent preemptions (minutes apart).
AI-written summary · about
chapters (11)
- 0:00Parcae and DNN training costs
- 1:21Spot instances for training challenges
- 2:26Checkpoint and redundancy limits
- 4:12Parcae’s preemption approach
- 5:19Laut definition and importance
- 8:15System design and core problems
- 9:39Predicting instance availability
- 12:05Lifeput optimizer strategy
- 13:30Migration strategies for preemptions
- 14:33Experimental results and gains
- 15:37Summary and open-source
related talks
15mBamboo: Making Preemptible Instances Resilient for Affordable Training of Large DNNs
12mPreview: ML Serving
16mPreview: Cloud Computing
24mFast training and near-linear scaling with DataParallel in Amazon SageMaker
14mHyCache: Hybrid Caching for Accelerating DNN Input Preprocessing Pipelines
29mRunning Massively Parallel Deep-learning Inference Pipelines on Kub... Suneeta Mall & Martin Abeleda
11mUnlocking the Power of Machine Learning for Cloud Workload Opt.
32mResearch Workloads on Preemptible VMs