Data Science Meets DevOps: Gitops with OpenShift
summary
Claims that GitOps can automate periodic retraining and deployment of models on OpenShift, and demonstrates a concrete pipeline: a DBSCAN clustering model trained on Google TestGrid data inside JupyterHub, saved as a pickle, committed to Git, then a git tag triggers Tekton pipelines to build a single ML container and push it to Quay; deployments are managed with Argo CD on an Open Data Hub instance. Cites an Algorithma survey: 55% never deploy models, only ~15% within a week, ~50% take one week–three months.
AI-written summary · about
chapters (11)
- 0:00Intro
- 1:55Traditional Software Development Lifecycle
- 3:57Machine Learning Deployment Process
- 9:55Tecton Pipelines
- 13:31Open Data Hub
- 15:46Open Data Hub Demo
- 22:26Notebook
- 23:44Data Visualization
- 27:34Push Changes
- 29:00Trigger TC Pipeline
- 32:14Conclusion
related talks
28mDevOps for Data Science With Kubernetes
16mGitOps for Machine Learning Pipelines
36mDataOps and MLOps: NLP Development and Deployment in Kubernetes made easy
30mContainer workflows for data science and ML
59mDevOps for Data Science
32mBridging DevOps and MLOps: Unifying Pipelines with KitOps and GitOps
57mDevOps for Machine Learning
23mAccelerating Model Deployment with MLOps