DevOps for Data Science? - automate the boring stuff and leverage the OSS ecosystem

on YouTube ↗29m

" summary

Argues that adopting MLOps automates data transformation, model training, testing, validation and deployment by wiring version control, CI/CD and cloud compute together; shows an end-to-end pipeline that uses DVC for data and model versioning, GitHub Actions to trigger training runs and post metrics to pull requests, and binder or Docker to reproduce notebook environments; demonstrates automated parameterized runs, reproducible artifacts on S3/Blob storage and smoother handoffs that eliminate the dozens of manual hoops between research and production.

AI-written summary · about

" chapters (5)

1. 0:00Start
2. 8:13THE ORIGIN OF DEVOPS
3. 9:36DEVOPS PRINCIPLES
4. 14:29CONTINUOUS INTEGRATION - SOFTWARE ENGINEERING
5. 15:35RECYCLE YOUR ECOSYSTEM

" related talks

• 59mDevOps for Data ScienceDamian Brady · NDC Sydney 2018
• 56mDevOps for Machine LearningDamian Brady · NDC Minnesota 2019
• 30mDevOps For Data TeamsTobias Macey · DevOpsDays Boston 2018
• 59mDevOps for Machine Learning ProjectsDmitry Pretrov, Ivan Shcheklein · The Fifth Elephant 2020
• 23mOpen Source DataOps and MLOps StrategiesLisa Cao · AI.dev: Open Source GenAI & ML Summit North America 2023
• 18mFrom DevOps to DataOps - Cloud based Software DevelopmentYuri Demchenko · OpenInfra Summit Berlin 2022
• 26mApplying DevOps practices in Data and ML EngineeringAntoni Ivanov · Conf42 Kube Native 2023
• 01mMLOps: Applying DevOps to MLDamian Brady · Data Love 2021