Untangling the Threads: Understanding Argo Workflows with Workflow Tracing
summary
OpenTelemetry tracing is wired into Argo via an OpenTelemetry Collector routing spans to Tempo and Prometheus, with traces from the workflow controller and pods plus optional otel CLI spans. Converting spans to metrics via a span metric connector produces alertable numbers (example: pending time climbs to ~6 minutes and a workflow stretched to 7 minutes when controller parallelism is 2). BuildKit exports detailed spans showing multistage parallel builds; controller reconcile delays are visible as the default 10‑second requeue. Cost and cardinality require selective tracing; release targeted for 3.7.
AI-written summary · about
chapters (15)
- 0:00Tracing in Argo Workflows
- 2:10OpenTelemetry architecture
- 3:53Workflow traces in Grafana
- 5:45Custom tracing with otel CLI
- 6:26Workflow phases analysis
- 8:14Spans to metrics conversion
- 10:48CI workflows tracing
- 13:40Build tracing with events
- 14:46Artifact transfer times
- 16:34Workflow controller interaction
- 18:01Workflow controller deep dive
- 19:23Local tracing setup
- 20:47Tracing challenges and plans
- 21:49Pipekit support and services
- 22:51Q&A
related talks
23mMastering Argo Workflows at Scale: A Practical Guide to Scalability Excellence
25mBeyond Production: End-to-End CI/CD Observability With OpenTelemetry
27mWhy Is It Taking so Long?! Shining the Light on Application..
29mUnleash ArgoCD Observability Superpowers
32mDisintegrated telemetry: The pains of monitoring asynchronous workflows
22mAnalysis in Action: Real-World Metrics Driven Argo Roll...
23mArgo at Scale: Navigating Complex Multi-Dimensional Deployments Across H...
26mDistributed Tracing FAQ