Explain How Kubernetes Works With GPU Like I’m 5
Talkat KubeCon + CloudNativeCon Europe 2025 · on Wednesday, 2025-04-02· at 11:15· Level 1 | Hall Entrance N10 | Room F·
summary
Homelab rebuild of the full stack from bare metal to running GPU-accelerated containers using an NVIDIA Jetson NUC, showing that with correct vendor drivers and matching CUDA versions (CUDA 11/12 compatibility matrix) plus the container toolkit and runtime hooks you can run LLMs like Llama/llama.cpp and DeepSeek R1 successfully with the GPU pegged at 100% in tests. Kubernetes integration uses device plugin registration with kubelet and node feature discovery labels, enabling allocation and simple time‑slicing (example: split one GPU into four shareable slots).
AI-written summary · about
chapters (15)
- 0:00Introduction
- 1:05Learn Kubernetes with GPUs at home
- 3:35Home lab with GPUs
- 6:08GPU-Kubernetes key components
- 7:52Device drivers and CUDA
- 11:24Container toolkit for GPU access
- 13:13Test GPU containers without Kubernetes
- 17:26Why device plugin is necessary
- 20:00Node and GPU feature discovery
- 23:17Orchestrating GPUs with Kubernetes
- 25:00Home lab setup
- 26:50Hybrid cloud-edge with GPUs
- 27:12Live demo: LLM on Jetson
- 28:37Limitations for small devices
- 29:18Resources and closing
related talks
30mRunning AI Workloads in Containers and Kubernetes
39m✨ GPU in My Containers!
26mState of the GPU(nion)
32mEverything you Need to Know about using GPUs with Kubernetes
32mScaling AI Workloads with Kubernetes: Sharing GPU Resources Across Multiple Containers
33mAI workloads on Kubernetes - How to maximize GPU utilization and cut costs
38mImproving GPU Utilization using Kubernetes- 32mScaling AI Workloads with Kubernetes: Sharing GPU Resources Across Multiple Containers