Tracing CI/CD with OpenTelemetry
summary
Istio uses OpenTelemetry to trace CI/CD pipelines end to end and reveals concrete bottlenecks: an e2e job emits 10,000 spans and runs five to six minutes with about one minute spent in kind cluster setup, a historical go test run takes ten minutes with two packages at two minutes each and a mysterious five minute gap, and a package with zero tests causes a 50 second block. Instrumentation requires starting/ending spans, propagating context (HTTP headers or environment variables), and adding tracing to CI platforms like Prow, Buildkite, GitLab or GitHub Actions.
AI-written summary · about
chapters (10)
- 0:00Tracing CI/CD intro
- 1:05Observability in CI/CD
- 2:08CI/CD observability limits
- 3:09Distributed tracing in microservices
- 4:11Tracing benefits in CI/CD
- 5:33CI/CD tracing improvements
- 6:39Logs insufficient for CI/CD
- 8:25Implement tracing in CI/CD
- 9:49Instrumenting CI/CD
- 10:30Summary & Q&A
related talks
32mDistributed Tracing in Action: Our Journey with OpenTelemetry
34mHow we Build CI/CD Observability with OpenTelemetry
28mEnd-to-end CI/CD observability with open source
24mStandardizing CI/CD Observability With OpenTelemetry: Insights From the CI/CD Observability SIG
08mLightning Talk: Kick-Start Distributed Tracing in Istio with OpenTelemetry
38mPractical introduction to OpenTelemetry tracing
45mSimplifying Instrumentation in complex systems at scale with Open Telemetry
43mObservability-Driven Development with Kubernetes