GPU Sharing for Machine Learning Workload on Kubernetes
summary
Implementation uses vGPU on vSphere plus a Kubernetes device-plugin + scheduler extension that represents a full GPU as 1000 units to allow fractional allocation and node label selection; a TensorFlow pod is scheduled to an 8GB vGPU in the demo. Measured overhead on RNN NLP training is ~4% training overhead. Targeted use cases are inference scale‑out (traffic camera license‑plate recognition) and multi‑tenant teaching clusters; current limits include one vGPU host per physical GPU and the code is still under development.
AI-written summary · about
chapters (18)
- 0:00Introduction
- 1:50What is AI
- 3:28Why Kubernetes
- 5:07Exclusive Assignment
- 6:46Virtual GPU
- 8:25Virtual GPU Configuration
- 10:04Virtual GPU Overhead
- 11:43GPU Sharing in Kubernetes
- 16:40Worker Node
- 18:19Fractional GPO
- 19:58Annotation
- 23:16Use Cases
- 24:55Demo
- 26:34Worker Knows
- 29:52Benefits
- 31:31Conclusion
- 36:28Fraction
- 38:07Device Plugin
related talks
39mKubernetes VMware User Group: Using GPUs with K8s on vSphere
38mMinimizing GPU Cost for Your Deep Learning on Kubernetes
33mAI workloads on Kubernetes - How to maximize GPU utilization and cut costs
38mImproving GPU Utilization using Kubernetes
31mKubernetes For GPU Powered Machine Learning Workloads In Academia
32mScaling Kubernetes Clusters for Generative Models: Managing GPU Resources for AI App...
38mMinimizing GPU Cost for Your Deep Learning on Kubernetes
38mKubernetes for AI Workloads – What Works and What… Doesn’t