Next-Generation Data Science Workflows Using Ray
summary
Shows Ray running on Kubernetes with JupyterLab via Open Data Hub and OpenShift: a tiny self-service workflow creates a Ray cluster CR from a notebook using a Python helper, ties lifecycle to the notebook with ownerReferences, and connects clients to a Ray 2.0 cluster. Demo trains a PyTorch model on Oxford Pets (3,600 images), runs hyperparameter tuning across 4 GPUs, inspects the Ray dashboard, then deploys with Ray Serve for REST inference; accuracy ~9% (random ~7%). Integrations include Ray Datasets, plasma/Arrow, Argo, IBM CodeFlare, and Project Thoth.
AI-written summary · about
chapters (11)
- 0:00Start
- 0:55Ray's Ecosystem Niche
- 2:02Ray's Compute Model
- 2:45Dependency DAG
- 4:18Ray's Core Data Model
- 5:19Native Ray Libraries
- 6:01Ray Community Integrations
- 7:05Ray is Cloud Native
- 8:12Open Data Hub Is...
- 28:16Ray + Cluster Autoscaling
- 29:21Roadmap
related talks
21mPowering Open Data Hub with Ray
25mPowering Open Data Hub with Ray
24mBuilding Complex Data Analytics Pipelines with Ray
25mData Science at Scale: From Notebook to Production with Airflow and Ray
26mRayDP: Build Large-scale End-to-end Data Analytics and AI Pipelines Using Spark and Ray
28mScaling AI & Machine Learning Workloads With Ray on AWS, Kubernetes, & BERT
26mIntroduction to Distributed ML Workloads with Ray on Kubernetes
01h 31mIntroduction to Ray for distributed and machine learning applications in Python