Automated Machine Learning Performance Evaluation
summary
Deploys a TensorFlow ResNet32 from a Google bucket into Seldon Core on Kubernetes, runs automated grid searches over CPU, RAM, replicas, threads and workers using Argo workflows, and benchmarks with vegeta (HTTP) and ghz (gRPC). Each run emits JSON via Argo logs, analysis uses Pandas to sort by throughput and percentiles, and an example shows three replicas yielding the highest request rate. The pipeline is designed to scale to hundreds or thousands of models and to surface optimal infra settings automatically.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 0:51C410 classifier
- 1:50What is seldomcore
- 2:53Deploying a model
- 6:10Extra complexity
- 8:04Best practices
- 8:50Benchmark types
- 9:27Benchmark tools
- 11:07Automating the evaluation
- 12:33Workflow managers
- 13:04Workflows
- 17:00argo workflow
- 19:21reusability
- 22:30output
- 24:09resources
- 24:57Wrap up
related talks
34mServing ML Models at Scale with Seldon and Kubeflow
36mAccelerating High-Performance Machine Learning at Scale i...
34mDemocratizing Machine Learning on Kubernetes [I]
36mAccelerating Machine Learning at Scale with
42mDeploying ML at Scale with Kubernetes and TFX
27mHarnessing Kubernetes for Scalable AI/ML Workloads- 42mDeploying ML at Scale with Kubernetes and TFX
40mScalable machine-learned model serving