Making the Most Out of Your Hardware Accelerators in a Kubernetes Cluster
Talkat Kubernetes Batch + HPC Day Europe 2023 · on Tuesday, 2023-04-18· at 14:45· Hall 7, Room A | Ground Floor | Europe Complex·
summary
Naive runs with one Tesla T4 plus one vCPU are slower than two vCPUs because the GPU spends long idle time waiting on CPU preprocessing; adding prefetching and vectorized preprocessing and parallelized reads reduces GPU idle time and makes the GPU-over-CPU configuration faster. Benchmarks run ResNet-50 on ImageNet inside Kubeflow compare parameter-server vs replicated strategies on two SLI GPUs and show similar images/sec for this small model. Examples use TensorFlow jobs on Kubernetes (and Flight).
AI-written summary · about
chapters (6)
- 0:00Start
- 5:10What do we run?
- 5:48Can this do better on a GPU?
- 12:34Setup
- 13:32GPU Strategy
- 23:39Configuring Flyte with GPU
related talks
34mSupercharge Kubeflow Performance on GPU Clusters
34mDemocratizing Machine Learning on Kubernetes [I]
34mBuilding GPU-Accelerated Workflows with TensorFlow and Kubernetes [I]
27mHarnessing Kubernetes for Scalable AI/ML Workloads
38mKubernetes for AI Workloads – What Works and What… Doesn’t
34mGPU as a Service Over K8s: Drive Productivity and Increase Utilization
36mAccelerating High-Performance Machine Learning at Scale i...
37mBuilding Distributed TensorFlow Using Both GPU and CPU on Kubernetes [I]