The State of SQL-Based Observability
summary
Argues that observability can be treated as a data problem by storing telemetry in a real-time OLAP SQL engine (example: ClickHouse) fed by OpenTelemetry collectors and optional Kafka buffering, queried from UIs like Grafana or Hyperdex. Gives production evidence: Uber and eBay deployments, ClickHouse Cloud ingesting close to one trillion rows per day and 43 PB under management, claiming extreme compression (reported as 2.4 TB on disk) and billion‑events/sec ingestion benchmarks. Lists tradeoffs: ops overhead, agent fleet management, and metric gaps addressed by a time‑series engine.
AI-written summary · about
chapters (13)
- 0:00Introduction to SQL observability
- 3:00SQL’s 50-year dominance in data
- 5:52History of observability systems
- 9:46SQL-based observability opportunity
- 10:28OLAP properties for observability
- 15:26Example SQL observability architecture
- 19:44Uber’s SQL observability at scale
- 21:31eBay’s open Telemetry integration
- 22:36ClickHouse’s self-observability
- 25:29Querying observability data in SQL
- 27:19Natural language to SQL queries
- 28:42Future SQL observability features
- 31:15Q&A on SQL observability
related talks
24mWhat Observability Can Learn From BI
52mObservability: it's not just an ops thing
28mScaling Observability for Real-Time Personalization Engines
33mWhat Developers Should Know About Observability
42mObservability Is Not Analytics!
32mSponsored Session: Observability and Agentic AI - From Observability to Answers...
36mObservability? It's a Data Analytics Problem, You Fool!
26mO11y in One: Exploring ClickHouse as a Unified Telemetry Database