Zero Downtime ML Deployments
summary
Claims a concrete deployment pattern: an ML monitoring service sits next to the prediction service, ingesting raw inputs and emitted predictions and computing metrics like RMSSE and conversion funnels. It detects prediction drift, data drift, feature drift and feature attribution drift, triggers shadow deployments and asynchronous checks, and favors graceful degradation over blocking inference. Mentions tooling such as vortex AI and Amazon Sage Maker and recommends comparing live traffic to historical ground truth before flipping models to production.
AI-written summary · about
chapters (9)
- 0:00Introduction to Zero Downtime ML Observability
- 1:07Understanding the Gap Between ML Engineers and SREs
- 2:43Challenges in Traditional Observability for ML Systems
- 4:45Addressing Silent Failures and Data Drifts
- 8:05Implementing Effective ML Monitoring Systems
- 10:37Types of Data Drifts and Their Impact
- 16:50Best Practices for ML Observability
- 18:52Tools and Techniques for Zero Downtime
- 20:31Conclusion and Final Thoughts
related talks
42mDeploying ML models and all the things that go wrong
13mMLOps: From AI Research to Production-Ready Systems
39mApplying deployment oriented mindset for building Machine Learning models
17mContinuous deployment for ML: The new software development lifecycle
29mDemystifying Machine Learning in Production: Reasoning about a Large-Scale ML Platform
39mContinuous Delivery for Machine Learning — The Future of MLOps
47mMachine Learning in Production: An Intro to MLOps.
39mAn SRE Approach to Monitoring ML in Production