From Model to Microservice - Machine Learning at Scale
summary
Shows a complete path from a scikit‑learn iris model to a live REST/gRPC microservice using Seldon Core deployed on Kubernetes, with a live demo returning a 99% iris-class probability. Explains handling of large (tens–hundreds GB) model binaries, 40+ model storage backends, configurable inference graphs for chaining/pre/post‑processing, Prometheus metrics export and built‑in drift detection and explanation endpoints for anchors/counterfactuals. Configuration is YAML CRD driven and the demo deploys and serves successfully.
AI-written summary · about
chapters (9)
- 0:00MLOps introduction
- 2:37Models in production
- 5:05Scaling ML challenges
- 9:01Kubernetes for ML
- 13:30Deploying model demo
- 18:01Inference graphs
- 20:47Model monitoring
- 22:51Adding explainability to machine learning predictions
- 24:00Q&A
related talks
34mServing ML Models at Scale with Seldon and Kubeflow
18mBeyond Inference: Bringing ML into Production
42mDeploying ML at Scale with Kubernetes and TFX
32mA CI/CD Framework for Production Machine Learning at Massive Scale (using Jenkins X and Seldon Core)- 42mDeploying ML at Scale with Kubernetes and TFX
35mCase Study: AI-as-a-Service on Kubernetes at Scale and In Production
37mSoftware Development for Machine Learning in Python
21mHarnessing Microservices for Scalable, Fault-Tolerant ML Systems