Everything you Need to Know about using GPUs with Kubernetes
summary
Kubernetes requires the NVIDIA driver installed on every GPU node and recommends building images with the CUDA toolkit but without vendor shared libraries, then requesting GPUs via nvidia.com/gpu. Legacy in-tree NVIDIA support is deprecated in 1.10 and removed in 1.11, replaced by device plugins (alpha in 1.8, beta in 1.10). Operators use a daemonset to install drivers and deploy a device plugin (NVIDIA or Google), set node labels/taints for GPU types, and can use resource quotas (added 1.10) and GPU metrics (added 1.9). GPU sharing, fine-grained type quotas, minikube support and topology-aware scheduling remain unresolved.
AI-written summary · about
chapters (12)
- 0:00Introduction to GPUs
- 1:12Challenges with containers
- 3:26Historical GPU support
- 6:01Device plugins overview
- 8:24Device plugin functionality
- 9:34User responsibilities
- 10:47Targeting specific GPUs
- 12:14Operator responsibilities
- 14:21GPU monitoring and quotas
- 16:03Optimizing node usage
- 19:18Future developments
- 22:39Q&A and conclusion
related talks
26mState of the GPU(nion)
30mExplain How Kubernetes Works With GPU Like I’m 5
38mImproving GPU Utilization using Kubernetes
33mAI workloads on Kubernetes - How to maximize GPU utilization and cut costs
39m✨ GPU in My Containers!
30mRunning AI Workloads in Containers and Kubernetes
36mProduction GPU Cluster with K8s for AI and DL Workloads
32mScaling AI Workloads with Kubernetes: Sharing GPU Resources Across Multiple Containers