Integrating Long-Term Storage with Prometheus [A]
summary
Prometheus adds a generic remote write/read interface that sends individual samples (not whole compressed chunks) over HTTP + protobuf (snappy), keeps limited batching and in-order per-series delivery to avoid memory blowups, and exposes relabeling and TLS/auth options. Real integrations: Chronix accepts remote-write only; Cortex provides horizontally scalable read+write, storing data in DynamoDB and S3 and allowing PromQL queries back. Demo shows remote read restoring deleted local history, proving the pipeline works end-to-end.
AI-written summary · about
chapters (18)
- 0:00Intro
- 0:25Local Storage
- 2:33Remote Storage
- 3:02Legacy Write Support
- 3:51Design Questions
- 4:56Decision
- 5:19Samples vs. Chunks
- 6:17Buffering and retries
- 7:32Protocol and Encoding
- 8:46Write Path Bridge
- 9:36Configuration
- 10:04Distributed or Centralized Eval?
- 11:13Read Path
- 11:51Future: Query Federation
- 13:13Note!
- 14:28Real World Examples
- 15:37Weave Cortex
- 21:30Build Your Own
related talks
37mIntro: Cortex
36mObservability with Prometheus and Beyond: Leveraging Cloud-native Technologies Outside of Cloud
26mIntegrating Prometheus and InfluxDB
36mPrometheus - Intro, Deep Dive, And Open Q+A
37mPrometheus Deep Dive
33mIntro: Prometheus
01h 19mIntro + Deep Dive: Prometheus
43mChronix as Long Term Storage for Prometheus