Deep Customized Kubernetes for Machine Learning in Tencent
summary
Tencent extends Kubernetes with a GPU topology-aware scheduler that picks card pairs to cut data transfer time roughly in half on tests, implements fractional GPU time-slicing (examples: 0.3 and 0.7 shares) and a lightweight server to mediate GPU commands for safe sharing, runs evaluations on Tesla P4 hardware, and optimizes Ceph-like object storage for high IOPS at scale (about 15 PB). A detailed implementation paper is due for release.
AI-written summary · about
chapters (8)
- 0:00Start
- 0:50GaiaStack ecosystem
- 1:19Why need a custom Kubernetes
- 2:47Highlights on GaiaStack
- 3:04GPU Topology scheduler
- 5:19GPU Share and Limit
- 14:59Tapp - a CRD of Kubernetes
- 16:47Galaxy - a powerful CNI plugin
related talks
31mKubernetes For GPU Powered Machine Learning Workloads In Academia
38mMinimizing GPU Cost for Your Deep Learning on Kubernetes
38mMinimizing GPU Cost for Your Deep Learning on Kubernetes
27mHarnessing Kubernetes for Scalable AI/ML Workloads
38mAll You Need to Know to Build Your GPU Machine Learning Cloud [B]
36mAccelerating High-Performance Machine Learning at Scale i...
34mDemocratizing Machine Learning on Kubernetes [I]
25mMaking the Most Out of Your Hardware Accelerators in a Kubernetes Cluster