Visualising and tracing requests through a cluster: Intergrating OpenTracing into OpenStack Swift
summary
Built a Swift WSGI middleware and trace mixin that injects OpenTracing context into requests, records spans across proxy/account/container/object pipelines and visualises them in Jaeger. Traces start three ways (HMAC signature, propagated context, or sampled every-N requests) and reveal surprising detail like 175 spans for account creation and six parallel connections for a 4+2 EC put. Overhead is marginal in small benchmarks, and practical issues include eventlet/greenlet context loss and response-iterator cleanup that needed pinning.
AI-written summary · about
chapters (20)
- 0:00Intro
- 1:53Overview
- 2:30What is Swift
- 7:47OpenTracing
- 12:02Whiskey
- 18:42Highlevel integration
- 19:07Demo
- 20:32Viewing the first trace
- 21:09Adding a container
- 21:59Transaction IDs
- 23:05Long Lines
- 23:32Junk objects
- 27:17Putting objects in
- 28:27Bulk Insert
- 30:47Extract Container
- 34:24Download Object
- 34:47Get manifest file
- 37:08Conclusion
- 41:24Questions
- 43:24Benchmarks
related talks
33mOpenTracing and Containers: Depth, Breadth, and the Future of Tracing
35mNavigating the Distributed Systems Execution Maze With OpenTracing
19mKeynote: OpenTracing and Containers: Depth, Breadth, and the Future of Tracing - Ben Sigelman
23mIntroduction to OpenTracing
34mWould You Like Some Tracing With Your Monitoring?
37mObservability, Distributed Tracing & the Complex World
43mPractical Introduction to OpenTelemetry Tracing
22mDistributed Tracing