ML Engineering Kubeflow
summary
Kubeflow provides a Kubernetes-native pipeline system that composes containerized tasks into reproducible ML workflows using Kubeflow Pipelines and TFX components, with an ML Metadata store that records artifact lineage (schema → trainer → evaluator → pusher). The demo compiles Python DSL into a pipeline, runs steps as Kubernetes pods, shows minutes‑scale runs where previously executed steps are skipped via caching reuse, and confirms models can be evaluated and automatically pushed if blessed.
AI-written summary · about
chapters (7)
- 0:00Introduction to Kubeflow
- 1:27Kubeflow project overview
- 6:44Kubeflow Pipelines purpose
- 9:34Kubeflow Pipelines architecture
- 17:03ML engineering best practices
- 22:26Demo of Kubeflow Pipelines
- 28:16Q&A
related talks
39mDeploy your ML Workloads on Kubernetes with Kubeflow
35mOrganizing data engineering around Kubeflow
14mKubeflow: Where ML Meets Kubernetes
05mLightning Talk: Build a Reproducible ML Workflow with Kubeflow Pipelines
36mMachine learning pipeline Kubeflow, let's start
54mScaling Machine Learning Pipelines in Cloud
30mKubeflow, Your ML Tools on Kubernetes
37mEnd-to-End ML Pipelines with KubeFlow and TensorFlow Extended (TFX)