Data Science Workflows Made Easy: Python-Powered Argo for Your Organization
summary
Shows how Hera (Python SDK) generates Argo YAML, runs scripts with the Hera Runner to allow native Python functions and complex return types, and uses hooks to inject auth, images, node selectors and org defaults. Demonstrates building images in CI, creating versioned workflow templates, and a live diabetes ML pipeline that runs locally and on Argo, outputs plots as artifacts, and scales from tens to hundreds (thousands possible) but requires a CI image build and careful etcd/workflow partitioning.
AI-written summary · about
chapters (5)
- 0:00Argo workflows with Python
- 6:41Hera and script templates
- 11:16Writing workflows in Python
- 16:11Live data science demo
- 21:32Key takeaways
related talks
22mOrchestrating Python Functions Natively in Argo Using Hera
27mBuilding a Data Science Platform with Argo Workflows
10mScaling Time Series Analysis With Argo Workflows: Patterns and Practi...
06mComparing Argo Workflows and Apache Airflow at Scale
12mLightning Talk: CI/CD for Data Pipelines with Argo Workflows
11mShipping Argo Workflows in Packages
26mComparing Argo Workflows and Airflow in a Distributed Environment
22mLeveraging Argo WorkflowTemplates Within a Data Platform