Observability? It's a Data Analytics Problem, You Fool!
summary
Claim: observability is a data analytics problem not just logs+metrics+traces. Concrete moves include adopting OpenTelemetry and its collector, adding continuous profiling and exemplars, rolling verbose logs into metrics, and storing rarely used records in cold tier. Surprising numbers: 99% of indexed observability data is never searched and ~75% of Kubernetes metrics go unused. Practical controls: use structured JSON logs and add trace ID to logs (or embed logs in spans) to enable correlation and efficient querying.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 1:19Traditional observability flaws
- 5:59Observability as analytics
- 8:19Beyond three pillars
- 11:35OpenTelemetry as a game changer
- 15:20Need for structured data
- 17:28Standardizing data models
- 19:09Enriching observability data
- 21:38Query visualization challenges
- 24:27Managing data volumes
- 30:07Key takeaways
- 31:55Q&A
related talks
35mBeyond Logs, Metrics and Traces. The Data Analytics Paradigm Shift in Observability — Dotan Horovits
36mMetadata: The Secret Sauce for Full Observability
42mObservability can help you understand change
32mRules of Observability: The Data Edition
42mObservability Is Not Analytics!
11mKubernetes Observability
32mUnderstanding Observability in Kubernetes
39mThe 5W’s of Observability with Practical Examples