Minimizing GPU Cost for Your Deep Learning on Kubernetes
summary
Describes an Alibaba-built native Kubernetes GPU sharing solution for Nvidia-based deep learning workloads, aimed at reducing idle-GPU waste. Covers the GPU sharing API design, how to enable scheduling of shared GPUs without changing the core Kubernetes scheduler, and how GPU isolation is integrated; includes a TensorFlow demo showing multiple jobs running on the same GPU and reports improved GPU utilization for development, debugging and inference.
AI-written summary · about
chapters (10)
- 0:00Start
- 2:48Why GPU is so fast?
- 3:39Scheduling GPUs on Kubernetes
- 6:39Why do we need to share GPU In Kubernetes?
- 7:29The Challenges of Sharing GPU in Kubernetes
- 9:21Design Thinking
- 14:09Architecture Overview(Cont.)
- 17:35Deploy GPU Sharing Capabilities in
- 19:18Use GPU Sharing in Kubernetes(Cont.)
- 25:19Summary & Next Steps
related talks
38mMinimizing GPU Cost for Your Deep Learning on Kubernetes
38mImproving GPU Utilization using Kubernetes
36mProduction GPU Cluster with K8s for AI and DL Workloads
33mAI workloads on Kubernetes - How to maximize GPU utilization and cut costs
34mDeep Customized Kubernetes for Machine Learning in Tencent
05mLightning Talk: Sharing a GPU Among Multiple Containers
24mImproving GPU Utilization and Accelerating Model Training with Scheduling Framework and NRI
37mBuilding Distributed TensorFlow Using Both GPU and CPU on Kubernetes [I]