Lightning Talk: Sharing a GPU Among Multiple Containers
Lightningat KubeCon + CloudNativeCon North America 2019 · on Tuesday, 2019-11-19· at 17:32· Exhibit Hall AB - San Diego Convention Center, Ground Level·
summary
Algorithmia shares a single GPU among multiple containers since 2016 and ships a custom scheduler extender, GPUShare Scheduler Extender, that works with Kubernetes device plugins and kubelet device advertising to colocate pods. GPUs cost up to 10x CPU instances and lack kernel cgroup limits and true GPU memory virtualization, so solutions use NVIDIA MPS to reduce CUDA context switching but risk global failures and memory overcommit. Production use requires realtime monitoring of per-pod GPU memory and throttling to avoid interference.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 0:16Background
- 0:39Why GPUs
- 0:57Cost
- 1:17Problem
- 1:37Limitations
- 2:45Kubernetes
- 3:18Custom Resource Management
- 3:46Performance Monitoring
- 4:17Conclusion
related talks
10mLightning Talk: K8s Issue #52757: Sharing GPUs Among Multiple Containers
32mScaling AI Workloads with Kubernetes: Sharing GPU Resources Across Multiple Containers- 32mScaling AI Workloads with Kubernetes: Sharing GPU Resources Across Multiple Containers
38mImproving GPU Utilization using Kubernetes
09mLightning Talk: Bringing Multi-Tenancy to GPU-Based Workloads
04mLightning Talk: Dynamic GPU Fraction and Sharing With Cloud Native Principle
30mRunning AI Workloads in Containers and Kubernetes
33mAI workloads on Kubernetes - How to maximize GPU utilization and cut costs