Doing Things Prometheus Can’t Do with Prometheus
summary
Argues Prometheus can handle HA, high-cardinality, long retention and basic anomaly detection if configured: run replicated Prometheus behind a proxy for failover and global queries, limit concurrent queries and set 50 million samples per query, shard or federate regionally, use disk-backed retention (production instance keeps ~250 days so far), build exporters with const metrics to avoid ghost series, compute SLOs from PromQL (example 99th‑pct → 95.8% over 24h), and use predict_linear for simple forecasts.
AI-written summary · about
chapters (30)
- 0:00Introduction
- 0:40Clickbait
- 1:24The problem with technology
- 2:32The goal
- 3:19What are metrics
- 4:18Metrics logs and traces
- 5:32Going beyond Prometheus
- 6:40Prometheus isnt special sauce
- 7:42Prometheus isnt a distributed system
- 8:50Gaps in metrics
- 9:20Prometheus replicas
- 13:42Prometheus cardinality
- 15:37Prometheus limits
- 18:45Aggregation
- 19:56Metrics
- 21:06Longterm storage
- 22:23New solutions
- 23:19Long term storage
- 24:09Machine learning
- 24:39Latency metrics
- 25:25Spikes odd events
- 25:53Predict linear function
- 27:19Cont metrics
- 29:27Prometheus Alert Manager
- 31:00SELLOS
- 31:36Setting an objective
- 32:18Latency
- 32:52Prometheus SubQuery
- 33:12Timebased SLO
- 34:45Conclusion
related talks
01h 19mIntro + Deep Dive: Prometheus
33mPrometheus Project Intro
36mPrometheus - Intro, Deep Dive, And Open Q+A
46mPrometheus as a Time Series Database
26mWhen Prometheus Can’t Take the Load Anymore
54mPrometheus – what’s new and what’s next?
10mPrometheus for Monitoring Metrics - Database Monitoring Tools
25mUnbreakable Prometheus: Leveraging Limits for Consistent Monitoring