Benchmarking Machine Learning Workloads on Kubeflow
summary
Kubebench is a benchmarking harness built on Kubeflow and Kubernetes that automates config generation, Argo workflows, shared storage and result aggregation, and plugs into Prometheus/Grafana. Version v0.3 demos local and distributed TensorFlow workloads, runs on CUDA 9.0 / TF 1.1 with GTX 1080ti testbeds, and reports 3–5% single‑GPU overhead and 5–10% distributed overhead. It keeps outputs in a shared filesystem, supports workflow YAML or parameterized launches, and exposes a dashboard for metrics and logs.
AI-written summary · about
chapters (10)
- 0:00Why Kubebench?
- 2:15What is Kubebench?
- 2:27Goals of Kubebench
- 3:52Tech Stack
- 4:49Architecture
- 7:28User's Perspective
- 9:02Where we are
- 14:44Local Training Benchmark
- 18:13Distributed Training Benchmark
- 21:33Testbed
related talks
51mMachine Learning using Kubeflow and Kubernetes
29mKubeflow: TensorFlow on Kubernetes
46mKubeflow, TensorFlow on Kubernetes
46mAccelerating Machine Learning DevOps with Kubeflow
31mKubeflow - Data Science on Steroids
32mA Practical Guide To Benchmarking AI and GPU Workloads in Kubernetes
36mIntroducing Kubeflow (w. Special Guests Tensorflow and Apache Spark)
36mKubeflow: Tensorflow on Kubernetes