Lightning Talk: K8s Issue #52757: Sharing GPUs Among Multiple Containers
summary
Kubernetes GPU sharing remains an open engineering problem since 2017 and now sees a practical stack: Dynamic Resource Allocation (DRA) for scheduler integration and HAMi as a software GPU virtualization/runtime. HAMi exposes fractional GPU cores and megabytes (examples claim 1% core and 1 MB granularity), works without MIG, supports NVIDIA/AMD/Ascend and ~10 chip types, and integrates with Volcano, Koordinator and Kueue. Expect concrete numbers: 4,000x memory footprint variance across AI workloads, typical production GPU utilization ~28%, and adopters reporting large utilization gains and cost reduction.
AI-written summary · about
chapters (7)
- 0:00GPU sharing in Kubernetes
- 1:28GPU sharing problem
- 3:19Limitations of GPU sharing
- 4:43H: GPU virtualization
- 6:51H in Kubernetes
- 7:35Production use cases
- 9:05Project status
related talks
05mLightning Talk: Sharing a GPU Among Multiple Containers
05mHami: Dynamic, Smart, Stable GPU-Sharing Middleware In Kubern...
38mImproving GPU Utilization using Kubernetes
30mRunning AI Workloads in Containers and Kubernetes
27mUnlocking Heterogeneous AI Infrastructure K8s Cluster: Leveraging the Power of HAMi
32mScaling AI Workloads with Kubernetes: Sharing GPU Resources Across Multiple Containers
09m⚡ Lightning Talk: Advanced GPU-Orchestrated Workflows and HPC Integrations on K8s for Distributed AI/ML at Scale- 32mScaling AI Workloads with Kubernetes: Sharing GPU Resources Across Multiple Containers