Minimizing GPU Cost for Your Deep Learning on Kubernetes
summary
A native Kubernetes GPU sharing solution for Nvidia-based deep learning workloads that reduces idle-GPU waste is presented. It covers a custom GPU-sharing API, scheduling GPU-shared workloads without modifying the core Kubernetes scheduler, and integrating GPU isolation into Kubernetes; a demo shows TensorFlow jobs from different users running concurrently on the same GPU and reports improved GPU utilization for model development, debugging and inference.
AI-written summary · about
chapters (9)
- 0:00Start
- 2:48Why GPU is so fast?
- 3:39Scheduling GPUs on Kubernetes
- 7:29The Challenges of Sharing GPU in Kubernetes
- 9:22Design Thinking
- 14:09Architecture Overview(Cont.)
- 17:35Deploy GPU Sharing Capabilities in
- 19:18Use GPU Sharing in Kubernetes(Cont.)
- 25:19Summary & Next Steps
related talks
38mMinimizing GPU Cost for Your Deep Learning on Kubernetes
38mImproving GPU Utilization using Kubernetes
33mAI workloads on Kubernetes - How to maximize GPU utilization and cut costs
36mProduction GPU Cluster with K8s for AI and DL Workloads
44mGPU Sharing for Machine Learning Workload on Kubernetes
37mBuilding Distributed TensorFlow Using Both GPU and CPU on Kubernetes [I]
34mDeep Customized Kubernetes for Machine Learning in Tencent
50mMaximizing GPU Utilization for Generative AI