Transparent GPU Sharing in Container Clouds for Deep Learning Workloads
summary
Claims a kernel‑intercept OS layer called TGS that enables transparent GPU sharing under Docker/Kubernetes with ~3000 LOC, adaptive rate control and transparent unified memory. Evaluated on A100/V100 with real traces, it cuts opportunistic job JCT by ~52% versus exclusive assignment, matches AntMan’s throughput without framework changes, and speeds opportunistic jobs up to 15× compared to NVIDIA MPS while keeping production throughput nearly intact.
AI-written summary · about
chapters (14)
- 0:00Intro
- 0:34Deep learning training jobs: important workloads in datacenters
- 1:06Deep learning training jobs in container clouds
- 1:49Low GPU utilization in production
- 3:39Application-layer solution: AntMan
- 6:11A more practical solution: TGS
- 7:38Sharing GPU compute resources
- 8:31Adaptive rate control of TGS
- 9:19Sharing GPU memory resources
- 10:08Transparent unified memory of TGS
- 11:07Evaluation setup
- 11:30Evaluation baselines
- 11:54Mixed workload job stream
- 13:53Conclusion
related talks
30mRunning AI Workloads in Containers and Kubernetes
38mMinimizing GPU Cost for Your Deep Learning on Kubernetes
38mMinimizing GPU Cost for Your Deep Learning on Kubernetes
38mAll You Need to Know to Build Your GPU Machine Learning Cloud [B]
32mScaling AI Workloads with Kubernetes: Sharing GPU Resources Across Multiple Containers
30mGPU Sharing at CERN: Cutting the Cake Without Losing a Slice!
09mLightning Talk: Bringing Multi-Tenancy to GPU-Based Workloads- 32mScaling AI Workloads with Kubernetes: Sharing GPU Resources Across Multiple Containers