Observability in the MLOps Lifecycle with Prometheus
summary
Shows concrete instrumentation: a FastAPI model served in Kubernetes with the Prometheus operator and a service monitor, a Grafana dashboard JSON that plots model metrics (prediction score, distribution over time) and standard CPU/GPU utilization, and simulated production traffic using Locust load tests. Demonstrates a live pipeline where data drift and edge cases cause a measurable drop in score, alerts via Prometheus/Grafana, and how Seldon and Flyte expose the same metrics for retraining or cost-root-cause work.
AI-written summary · about
chapters (6)
- 0:00Introduction to ML reliability
- 2:27ML lifecycle and challenges
- 4:25Production monitoring factors
- 7:18Core metrics for observability
- 10:59FastAPI demo and implementation
- 19:33Scaling with MLOps frameworks
related talks
05mLightning Talk: Machine Learning Observability with Prometheus
37mLowering the barrier for ML monitoring
28mImprove Monitoring and Observability for Kubernetes with OSS Tools
56mFull-stack observability and application monitoring with
30mFrom Monitoring to Observability: Left Shift your SLOs with Chaos
46mLeveraging Prometheus Service Monitor and Pod Monitor for Enhanced Kubernetes Monitoring
27mObservability with Prometheus and beyond
34mMonitor My Socks: Using Prometheus in a Polyglot Open Source Microservices