Lightning Talk: Optimizing Latency in Argo Workflow Submissions
summary
Measured submission latency grows with the number of WorkflowTemplate references because each reference triggers repeated Kubernetes GETs, producing a clear template get spike and API server/etcd load that worsens under concurrent submissions. Workarounds tested include rate-limit overrides, submitting CRs to bypass Argo server validation, client-side HTTP caching, and server-side caching; the most effective is adding an in-memory cache to validation via the list-watcher informer pattern. Splunk has a pending PR implementing this and reports large latency reductions.
AI-written summary · about
related talks
22mLeveraging Argo WorkflowTemplates Within a Data Platform
25mOne Engine To Rule Them All: Unifying Cloud Workloads With Argo Workflows
36mArgo Workflows: Intro, Update and Deep Dive
12mLightning Talk: Migrating 1,300 Jenkins Jobs To Argo Workflows: Lessons From a…
09mLightning Talk: How to Avoid a Kubernetes Doom Loop
12mLightning Talk: CI/CD for Data Pipelines with Argo Workflows
06mComparing Argo Workflows and Apache Airflow at Scale
27mThe Life of an Argo Workflows Salesperson