A Tale of Two Worlds: Canary-Testing for Both ML Models and Microservices
summary
Demo shows Kubernetes + Vamp (fem/ctl) driving canary releases for both a microservice and its ML model, wiring metrics from Prometheus/Elasticsearch and a UI that tracks favorites per minute. Traffic moves in steps (example: 50% of traffic to the new model), health checks trigger auto rollback when the canary fails, fix is deployed and traffic shifts to v2. Metadata and lineage live in ArangoDB metadata graph so teams query which datasets fed which models. Experiment wins when favorites per minute increases and the model is promoted into production.
AI-written summary · about
chapters (27)
- 0:00Introduction
- 0:41Jrgs background
- 1:10A Tale of Two Worlds
- 1:48The Two Worlds
- 2:00Microservices
- 4:27Shadow deployments
- 5:29Deployment vs Release
- 6:33FEM Stack
- 7:11ML Models
- 8:44Why is ML so successful
- 9:09What data scientists want to do
- 10:03Model training
- 10:30Model serving
- 10:41Tensorflow extended
- 13:38Update a Model
- 14:42Choosing Models
- 14:55Metadata
- 16:38Unstructured Data
- 17:26Rango DB
- 18:14One World
- 18:59Discovery Service
- 19:35Motivation
- 20:02Demonstration
- 26:16Discovery
- 27:29Experiments
- 30:00Other scenarios
- 31:31Conclusion
related talks
20mShip your Machine Learning Application #bbuzz
21mI Don’t Always Test My Code, but When I Do It’s in Production
42mBuilding a Canary Testing Framework
25mObservability In ArgoCD/Rollouts Using Streaming ML For Reducing MTTR
33mDeploying and Scaling Spark ML and Tensorflow AI Models
31mApplied MLOps to Maintain Model Freshness on Kubernetes
31mMonitoring machine learning models in production
24mObservability in the MLOps Lifecycle with Prometheus