Tutorial: Introduction to Kubeflow Pipelines
summary
Creates a GKE cluster, installs Kubeflow and Kubeflow Pipelines, then compiles a pipeline (Python SDK -> tar -> Argo) that trains a TensorFlow model from a checkpoint, serves it, and launches a web front end that generates titles for GitHub issues. Pipeline requests 4 GPUs, ~48 GB RAM so auto-provisioning adds GPU nodes on demand and removes them after training; storage uses GCS buckets and Cloud Shell is used for orchestration. The end-to-end app runs when components finish provisioning; premature auto-provisioning sometimes breaks deployment.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 5:20Getting Started
- 13:03Who is this for
- 16:20Create a cluster
- 21:04Why Kubeflow
- 23:35Challenges
- 25:08Cluster Setup
- 32:47Goals
- 36:06AutoProvisioning
- 37:46Central Dashboard
- 41:11Logging
- 43:36Dashboard
- 45:21Delete Deployment
- 47:19Access Central Dashboard
- 48:09Access Services Dashboard
- 53:11Troubleshooting
related talks
54mScaling Machine Learning Pipelines in Cloud
51mMachine Learning using Kubeflow and Kubernetes
36mIntroducing Kubeflow (w. Special Guests Tensorflow and Apache Spark)
31mKubeflow - Data Science on Steroids
21mIntroducing Kubeflow (w. Special Guests Tensorflow and Apache Spark)
36mMachine learning pipeline Kubeflow, let's start
36mKubeflow: Tensorflow on Kubernetes
41mAlice in the world of machine learning