How Tracing Uncovers Half-truths in Slack’s CI Infrastructure
summary
Claims a concrete outcome: using SlackTrace and SpanEvents plus Honeycomb, Kafka and Presto to instrument CI uncovers root causes and drives fixes. Shows examples like Git LFS slowing a QA fleet, traces that surface memory spikes, and a macro KPI dropping from ~50% flake rate to ~5%. Describes how shared span dimensions, cross-service traces, and replayable queries get built into workflows and consistently find actionable causes for flaky tests.
AI-written summary · about
chapters (12)
- 0:00Intro
- 4:56Developer Productivity
- 9:13Span Event Structure
- 14:55Whats Next
- 15:59Shared Dimensions
- 16:28Use Cases
- 16:52The Root Challenge
- 18:05The Results
- 18:39Fuzzy Service Boundaries
- 20:06Incident Command System
- 21:20Testing Changes
- 22:22Summary
related talks
34mSlackTrace: A New Tracing Tool
46mHow to Properly Blame Things for Causing Latency
50mDistributed Tracing in the Wild
43mObserving from Incidents
34mDistributed Tracing, Lessons Learned
24mDistributed Tracing : New DevOps Foundation.
34mReduce Complexity with Distributing Tracing
42mFounder/CTO Perspectives: The Future of Distributed Tracing