Vending Machine for Accelerating Data Science Experiments
summary
Concrete recipe for a self‑service ML “vending machine”: deploy Kubernetes + JupyterHub + Ray + Carpenter + Crossplane, create per‑experiment namespaced Ray clusters and provision S3 buckets and IAM roles automatically, and let Carpenter spin up GPU node pools in seconds using price‑aware spot/on‑demand choices. Demo runs two simultaneous experiments (CPU and GPU) that auto‑scale to zero, mount S3 with S3 Mountpoint, and successfully complete distributed transforms and image classification without manual infra steps.
AI-written summary · about
chapters (19)
- 0:00Introduction
- 1:49AIML priorities 2022
- 3:13ML workload needs
- 4:19Ray framework overview
- 6:53Challenges of running ML workloads on CPUs vs GPUs
- 8:16GPU vending machine
- 9:18Cluster autoscaler limits
- 10:03Carpenter intro
- 11:51Workload-based GPU types
- 13:43Helm chart config
- 14:49Concurrent experiments
- 16:39Demo setup
- 21:55Kubernetes API extension
- 23:26S3 Mountpoint
- 23:47Live vending demo
- 34:40Team B batch inference
- 40:45CPU cost savings
- 41:27Conclusion
- 42:11Q&A
related talks
17mProductionizing Data Science Workloads
45mAccelerate Model Training with an Easy to Use High-Performance...
13mDesigning an Autonomous Workbench for Data Science on AWS
31mNext-Generation Data Science Workflows Using Ray
28mRay on Public-Cloud k8s - experiments, lessons & best practices
38mAll You Need to Know to Build Your GPU Machine Learning Cloud [B]
21mLeveraging Ray to Enhance Machine Learning Models in Finance
37mA Day in the Life of a Data Scientist. Conquer ML Lifecycle on Kubernetes