Model Monitoring: Detecting and Analyzing Data Issues
summary
Models fail silently when upstream data changes or becomes all zeros, so production needs monitoring that detects anomalies and drift rather than just uptime. The proposal is a lightweight logger inside application that precomputes stats, uses sketching algorithms to handle high throughput with low overhead, visualizes time series and feature distributions with distribution distance metrics, and runs time-windowed detection for short‑term anomalies and long‑term drift so teams can lower MTTR.
AI-written summary · about
chapters (10)
- 0:00Intro
- 0:45Model serving observability
- 2:28Data issues: a new failure mode
- 4:27Data dependencies
- 5:05Model deployments
- 5:52Continuous data monitoring
- 7:11Model monitoring system
- 9:45Data analysis dashboard
- 10:30Automatic issue detection
- 11:11Conclusion
related talks
19mModel Performance Monitoring - A Practitioner's Perspective
26mML Monitoring & Observability
18mReliable Maintenance of Machine Learning Models
39mAn SRE Approach to Monitoring ML in Production
19mMinimize Risks and Accelerate MLOps with Model Performance Monitoring and Explainability
30mBetter data quality with ML & visualization
18mAI Observability: How To Fix Issues With Your ML Model
36mThe ML Monitoring Flow for Models Deployed to Production