Thanos: Prometheus at Scale!
summary
Shows how Thanos turns many Prometheus instances into a single, global metric system by adding a sidecar + Store API to each Prometheus, a stateless querier that deduplicates HA replicas, a store-gateway that reads blocks from object storage (S3/GCS/Azure/Alibaba/Swift), and a compactor that performs compactor downsampling and block consolidation. Claims: global queries and HA work transparently, long‑term retention scales via object storage, millions of series and year‑long queries become practical and much faster in production.
AI-written summary · about
chapters (19)
- 0:00Intro
- 1:53Do I Need Monitoring
- 3:42Monitoring Signals
- 5:11Troubleshooting Journey
- 6:35Instrumenting an Application
- 9:07metrics Page
- 11:10Prometheus: Pull Model
- 19:09Prometheus at Scole: Functional Sharding
- 22:33Prometheus at Scale: Challenges
- 25:13Prometheus at Scale: HA Load Balancing
- 25:57Prometheus at Scale: Retention
- 28:21Downsampling
- 32:21Transforming Monitoring: Seven Steps
- 32:50Solving Global View + HA
- 36:17Long-Term Retention
- 41:48Response caching
- 42:52Query performance & cost reduction
- 43:50Bonus: Streaming
- 46:55Summary
related talks
22mMonitoring the World: Scaling Thanos in Dynamic Prometheus Environments
27mHow Thanos Almost Snapped $100,000 from Our Infra Budget
32mAdopting Thanos Gradually Across All of LastPass Infrastructures
18mArchitecting Growth: Scaling Tactics for Prometheus Metrics...
31mTaming 50 Billion Time Series: Operating Global-Scale Prometheus Deployments on Kubernetes
10mMonitoring Production Clusters at Scale
06mLearnings from Running Prometheus on Google Cloud Scale
27mFrom Supercomputing to Serving: A Case Study Delivering Cloud Native Foundation Models