Reproducible and maintainable data science code with Kedro
summary
Claim: kedro turns messy Jupyter experiments into reproducible, maintainable, modular pipelines by enforcing config, a data catalog, and a pipeline abstraction with connectors to pandas, Spark and cloud storage. Upload cites NYU finding 860,000 notebooks, 24 run and only 4% reproduce results. Concrete outcome: a project template, visualized DAG and runnable pipelines, plus dataset versioning and catalog.load/save so the refactored notebook actually runs end to end.
AI-written summary · about
chapters (4)
- 0:00Start
- 2:07What is your end goal?
- 3:17The challenges of creating machine learning products
- 6:47Concepts in Kedro
related talks
22mImprove Your ML Projects: Embrace Reproducibility and Production...
49mKedro + MLflow - Reproducible and Versioned data pipelines at scale
40mData Science Best Practices with Kedro
33mHow to build production-ready data science pipelines with
35mUntitled12.ipynb
01h 28mMaintainable code in data science
35mOrganizing data engineering around Kubeflow
33mCompliant Data Management and Machine Learning on Kubernetes