Ray on Public-Cloud k8s - experiments, lessons & best practices
summary
Runs Ray + Ludwig Tune hyperparameter search across 12 tabular datasets with up to 300 epochs per job and some single runs equivalent to 30 days of compute. Baseline uses EC2 g4dn.extra large VMs and S3 — simple to configure but manually manages instance counts and avoids spot volatility. Kubernetes on OKE adds the Ray autoscaler and OCS (S3‑compatible) storage for better tracking and standardization. Nodeless Kubernetes automates instance selection and cuts cost but injects a 2–7 minute pod provisioning delay.
AI-written summary · about
chapters (14)
- 0:00Intro
- 1:32Introduction to Ray
- 3:51Machine Learning on Ray & the Ludwig ML Toolbox
- 6:28Machine Learning Model Search Experiments We recently used the Ludwig Benchmarking Toolkit on the Ludwig Ray platform
- 7:27ML Experiment Overview: Datasets Trained deep learning models with hyperparameter search for 2 tabuler datasets
- 9:04ML Experiment Overview: Model Configurations
- 10:35ML Experiment Overview: Execution
- 12:50VM Cloud Execution Environment: Description
- 14:29VM Cloud Execution Environment: Pros
- 18:36Cons: Operational Complexity, Cost
- 21:05Nodeless Kubernetes Cloud Execution Environment
- 24:47Lessons Learned and Best Practices Ray-based ML Experiments can run on a variety of Cloud Environments • Choosing execution environment up front is valuable, given switching costs, eg
- 26:30Lessons Learned and Best Practices, continued
- 27:37Acknowledgements
related talks
28mScaling AI & Machine Learning Workloads With Ray on AWS, Kubernetes, & BERT
26mIntroduction to Distributed ML Workloads with Ray on Kubernetes
39mSailing Ray Workloads with KubeRay and Kueue in Kubernetes
32mRay in Production: Engineering Lessons from Spotify's Hendrix Framework
19mScaling Frontier AI with Ray on Kubernetes
32mAn Overview of CloudKitchens's Ray-Powered ML Platform
31mNext-Generation Data Science Workflows Using Ray
28mEfficient Deep Learning Training with Ludwig AutoML, Ray, and Nodeless Kubernetes