Distributed Tracing, Lessons Learned
summary
Company builds a custom tracing platform in F# to track long asynchronous flows through Kafka and Event Sourcing, emitting compact telemetry events (trace ID, operation, tags) to local log files read by Filebeat into Kafka. An indexer called x-ray service stores traces in Cosmos DB and uses the cosmos DB change feed to produce five‑minute topology snapshots sent to Splunk for SLA alerting. Library auto‑populates tags, supports multiple carriers, and the system works well for dependency/SLA queries after iterating on scope and onboarding.
AI-written summary · about
chapters (8)
- 0:00Start
- 2:01Core Contributors
- 2:20Venn Sourcing
- 17:08P Tags and M Tags
- 21:11Computation Expressions
- 26:30System Diagram
- 30:19Future of Tracing
- 31:51Questions
related talks
45mTracing: A journey to tactical insights
45mMastering Distributed Tracing: Insights from Building a Platform from Scratch
24mDistributed Tracing : New DevOps Foundation.
31mFrom Alerts to Insights: Performing Trace-Based Causation at Scale
49mDistributed Tracing: How the Pros Debug Concurrent and Distributed Systems
29mIntroduction to Distributed Tracing
01h 02mAdding Observability to Distributed Systems
26mDistributed Tracing FAQ