⚡ Lightning Talk: Configuring Volumes for Parallel Workflow Reads and Writes
Lightningat ArgoCon Europe 2023 · on Tuesday, 2023-04-18· at 12:05· Elicium Building | Elicium Ballroom 2 [Argo Workflows]·
summary
Using Argo Workflows with Argo CD plus an NFS server provisioner converts EBS or node ephemeral disks into ReadWriteMany volumes that pods can dynamically provision via PVC templates, enabling true parallel reads/writes. A crude 40 GB test writes 40 GB across steps and shows S3 taking ~7 minutes while NFS completes in 20 seconds. Deploy the provisioner twice (EBS-backed for semi‑persistent, ephemeral for transient), mount PVCs in workflows, and observe CI times drop from ~15–20 minutes to about 3 minutes.
AI-written summary · about
chapters (8)
- 0:00Start
- 1:02The Problems
- 1:32Storage Options
- 3:12How does it work for semi-persistent data?
- 3:39How does it work for transient data?
- 4:19Comparing Cost
- 4:52Testing Workflow Speed
- 6:25Conclusion
related talks
33mScaling to Thousands of Data & CI/CD Pipelines Using Argo and Virtual...
12mLightning Talk: Using Argo Workflows to Curate Chaos Engineering wi...
25mOne Engine To Rule Them All: Unifying Cloud Workloads With Argo Workflows
05mProject Lightning Talk: Argo CD at Scale: 5 Features You Should Not Miss!
09mLightning Talk: Optimizing Latency in Argo Workflow Submissions
10mLightning Talk: Does Cloud Elasticity Pay off for HPC Workloads?
14m300 Hours of Repetitive Work in Just 3 Hours by Leveraging Argo Workflow...
27mThe Life of an Argo Workflows Salesperson