Distributed Python with Ray-Hands on with the Ray 2.0 APIs for scaling Python Workloads
summary
Hands-on demos use Ray 2.0 Core APIs and Ray AIR to scale Python from laptop to cluster: remote functions as tasks, remote objects as futures, and stateful actors run on a local 10-core cluster and a dashboard shows live metrics. Notebooks on GitHub demonstrate batch inference (model shards + actor pool), a parameter-server–style trainer tree, and map-reduce patterns; a Fibonacci microbenchmark drops from ~1.9s locally to ~0.34s distributed. Code is runnable, models and actor pools load into the object store and predictions complete.
AI-written summary · about
chapters (11)
- 0:00Ray introduction
- 4:56Why Ray emerged
- 14:42Ray architecture
- 27:50Real-world Ray use cases
- 35:33Setting up Ray
- 48:38Batch inference with Ray
- 55:07Ray object references
- 1:01:49Ray actors
- 1:13:48Parameter server pattern
- 1:21:37Tree actor pattern
- 1:28:19Scaling inference with actors
related talks
57mIntroduction to Ray for distributed and machine learning applications in Python
02h 51mDistributed Python with Ray Hands on with the Ray Core APIs
01h 31mIntroduction to Ray for distributed and machine learning applications in Python
26mRay: A System for High-performance, Distributed Python Applications
31mScaling Python to thousands of nodes with Ray
31mNext-Generation Data Science Workflows Using Ray
39mScaling Python for Machine Learning: Beyond Data Parallelism
29mLearning to Scale Data Science, Machine Learning, and Pandas with Ray and Modin