Navigating Distributed Traces
summary
Built interactive tooling on top of Jaeger that turns millions of traces into actionable views: aggregated latency histograms, dependency graphs, and a “jump to representative traces” link. Uses variance decomposition to find services whose high variance amplifies up the stack. Demo shows a real root cause: a 0.001% timeout pattern traced to software networking blocking (100 ms gaps) rather than DB or GC. Tools are incorporated into Jaeger and prove useful for pinpointing and fixing both performance and outage drivers.
AI-written summary · about
chapters (16)
- 0:00Start
- 1:04How I Found Distributed Tracing
- 3:18What Distributed Tracing Is
- 9:20Customer Impact
- 12:30Auto Remediation
- 13:08How Do We Use Tracing for Performance Use Cases
- 17:39Dependency Graphs
- 24:22Latency Measurements
- 29:26Latency Decomposition
- 30:31Variance Decomposition
- 31:32Moving Strongly Connected Services around Data Centers
- 37:08Latency Histograms
- 38:44Trace Data
- 40:11Networking Events
- 42:01Garbage Collection
- 49:58Resistance
related talks
34mWould You Like Some Tracing With Your Monitoring?
35mJaeger Intro
37mDeep Dive: Jaeger – Steve Flanders
50mThe hidden gems of distributed tracing
57mDistributed Request Tracing in .NET
32mA Picture is Worth 1,000 Traces
20mThe (Full)stack Trace: Understand Your App with Distributed Tracing
37mDeep Dive: Jaeger