When Kubernetes Meets MLflow: Orchestrating MLOps Processes the Cloud Native Way
summary
Demonstrates a concrete pipeline: MLflow configured and controlled by Kubernetes operators so the cluster becomes the reconciliation plane. Uses MLflow REST/Swagger to generate a Kubernetes operator from MLflow OpenAPI, deploys via Helm, wires GitHub workflows to trigger runs, captures metrics and pushes chosen runs to the MLflow model registry. Publishes a self‑service portal that automates project scaffolding: GitHub repo creation, template clone, MLflow experiment, first run and model registration; the prototype is open sourced and works end‑to‑end in customer trials.
AI-written summary · about
chapters (13)
- 0:00Kubernetes and MLflow integration
- 1:26MLOps process challenges
- 3:12Kubernetes operators as control planes
- 4:37Composable MLOps and self-service
- 7:05Kubernetes operators for automation
- 9:52Internal developer platform roles
- 11:19Data scientist workflow with MLflow
- 13:07Kubernetes controller for MLflow
- 16:20Automating MLflow experiments
- 18:55Portal for data scientist onboarding
- 20:43Golden paths for data scientists
- 21:49MLOps takeaways
- 23:16Cost allocation and future-proofing
related talks
14mMLOps: DevOps for AI with KubeFlow Pipelines
22mBuilding a MLOps Platform with Kubeflow on GKE
24mAutomating ML Pipelines With Kubernetes And Airflow
39mDeploy your ML Workloads on Kubernetes with Kubeflow
01mE2E ML Platform on Kubernetes with just a few clicks
14mKubeflow: Where ML Meets Kubernetes
06mEvolving with Kubernetes: Embracing Model Ops
29mML Engineering Kubeflow