"Adding Observability to Distributed Systems"
summary
Proofpoint’s cloud product evolves from a startup mess into a debuggable system by adding distributed tracing built on existing logs and metrics, using Kafka, Jaeger/Zipkin and the Elastic APM OpenTracing bridge. The codebase grows to ~100 microservices across thousands of VMs and 200x more data, so tracing plus trace IDs and automation cuts incident diagnosis from days to ~30 minutes. Practical details: instrument critical paths, sample traces, store/visualize spans, link SLIs to deployments.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 5:00How confident are you at this moment
- 7:40Observability Venn diagram
- 12:30Why build Observability
- 14:15Service Level Indicators
- 20:37Metrics
- 28:28Logging
- 33:52Distributed Tracing
- 38:43How Tracing Works
- 40:27Spans
- 44:29Business Value
- 47:38Tracing
- 52:01Tracing Standards
- 54:12Sampling Traces
related talks
01h 02mAdding Observability to Distributed Systems
49mImproving Observability of Distributed Systems
24mDistributed Tracing : New DevOps Foundation.
45mMastering Distributed Tracing: Insights from Building a Platform from Scratch
12mMastering Observability in Distributed Systems
16mThe State of Observability
39mConnecting the dots: observability in large e-commerce environments
41mA peek into observability from testers lens