Powering Open Data Hub with Ray
summary
Shows how Ray becomes an on-demand compute backend for Open Data Hub Jupyter notebooks by using ray.connect client-server instead of node-local ray.init and by provisioning Ray clusters via single user profiles and service templates (Kubernetes ConfigMaps). Demonstrates a working setup on OpenShift and the Mass Open Cloud (cluster limited to five workers, 3 GB each), runs an XGBoost training with two Ray actors, and returns a low-error model plus a reachable Ray dashboard.
AI-written summary · about
chapters (14)
- 0:00Introduction and agenda
- 1:12Ray at a high level
- 4:39Ray’s compute and data model
- 7:28Ray client-server connection
- 8:11What is Open Data Hub
- 10:42Spark integration in ODH
- 11:26Replacing Spark with Ray
- 12:27Demo setup and architecture
- 13:32Running XGBoost with Ray
- 15:39Visualizing results
- 16:45Benefits of Ray in ODH
- 17:49Mass Open Cloud demo
- 18:31Operate First project
- 19:14Future work and Q&A
related talks
25mPowering Open Data Hub with Ray
31mNext-Generation Data Science Workflows Using Ray
28mScaling AI & Machine Learning Workloads With Ray on AWS, Kubernetes, & BERT
29mAdopting Ray @ LinkedIn
28mRay on Public-Cloud k8s - experiments, lessons & best practices
26mIntroduction to Distributed ML Workloads with Ray on Kubernetes
26mRayDP: Build Large-scale End-to-end Data Analytics and AI Pipelines Using Spark and Ray
26mData Processing on Ray