Data Observability: From 1788 To 2032
summary
Argues that data observability is visibility into data state using four pillars—metrics, metadata, lineage, logs—and claims Metaplane can connect data stack in less than 10 minutes to run automated anomaly detection over tables, detect incidents, compute downstream impact, and suggest root causes by combining lineage and access logs. Demonstration fails live, but product is described as producing alerts into Slack, measuring reduced MTTR and improved trust via quarterly NPS and SLOs.
AI-written summary · about
chapters (16)
- 0:00Introduction and overview
- 1:27Ancient data and origins
- 3:57Evolution of data storage
- 5:44Modern data use cases
- 7:28Challenges of data complexity
- 8:32Software observability inspiration
- 10:38Control theory and observability
- 12:04Defining data observability
- 14:12Four pillars of data observability
- 16:16Example data stack walkthrough
- 17:39Incident monitoring use case
- 20:08Setting data observability goals
- 22:35Responsibility and incentives
- 23:39Tools and implementation
- 25:25Customer success example
- 26:27Future of data observability
related talks
41mData Observability - System Health to Business Health
33mBeyond Monitoring: The Rise of Observability Platform
35mBeyond Logs, Metrics and Traces. The Data Analytics Paradigm Shift in Observability — Dotan Horovits
40mDemystifying Observability and Monitoring
48mAPI Observability by Design: Building Systems That Tell Their Own Stories
19mObservability for Data Pipelines: Monitoring, Alerting & Tracing Lineage
01mObservability
42mObservability Is Not Analytics!