Reverse-Engineering PromQL Usage: A Proxy’s Tale
summary
Built proxy sits between clients and any Prometheus-compatible backend and forwards queries while asynchronously persisting raw PromQL, query type/step, label matchers, duration, status codes and engine fields like total and peak samples into Postgres or SQLite. It fingerprints expressions to roll up usage, integrates metric-usage scans from the matrix-usage tool, and supports per-instance or sidecar deployment as a stateless service. Claims low overhead, zero-change to Prometheus, captures raw PromQL+samples+fingerprint, and reports running in production at 40k–50k queries/s with ~1 GB RAM per instance.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 1:45Expensive PromQL queries
- 3:10Query analysis methods
- 4:36Prom Analytics proxy
- 6:43Capturing query data
- 7:47Demo and system health
- 10:02Query fingerprints and execution
- 13:36Metrics usage correlation
- 15:47Metrics catalog and visualization
- 18:39Future plans
- 21:17Q&A: Long queries
- 22:44Q&A: Query sources
- 23:46Q&A: Multi-tenancy
- 24:53Q&A: Scalability
- 27:00Q&A: Dedicated UI
related talks
29mInside a PromQL Query: Understanding the Mechanics
34mBlazin’ Fast PromQL
10mPrometheus for Monitoring Metrics - Database Monitoring Tools
33mIntro: Prometheus
01h 19mIntro + Deep Dive: Prometheus
35mDoing Things Prometheus Can’t Do with Prometheus
37mAI in SRE: Unlocking Prometheus Insights with Natural Language
32mPrometheus Intro and Deep Dive