Introduction to Ray for distributed and machine learning applications in Python
summary
Hands-on coding shows Ray Core 2.0 turning Python functions into distributed tasks, actors, object store primitives that run on laptop or cloud and integrate with PyTorch XGBoost HuggingFace and Kubernetes. Notebooks run Monte Carlo Pi and Fibonacci examples that drop serial runs (≈1.9s) to concurrent execution (≈42ms) and demonstrate data sharding, zero‑copy shared memory, and model training/tuning libraries. Exercises and scripts are provided and the examples complete successfully, illustrating real speedup and practical orchestration.
AI-written summary · about
chapters (13)
- 0:00Welcome
- 1:02Speaker intro
- 3:08What is Ray
- 8:27Ray architecture
- 15:25Scaling patterns
- 21:44Q&A actors
- 26:15Ray API demo
- 33:46Ray for distributed
- 35:10Ray in large language model training
- 42:10Summary
- 42:54Q&A integrations
- 47:02Ray setup demo
- 51:51Pi with Ray
related talks
01h 30mDistributed Python with Ray-Hands on with the Ray 2.0 APIs for scaling Python Workloads
02h 51mDistributed Python with Ray Hands on with the Ray Core APIs
26mRay: A System for High-performance, Distributed Python Applications
29mLearning to Scale Data Science, Machine Learning, and Pandas with Ray and Modin
31mScaling Python to thousands of nodes with Ray
39mScaling Python for Machine Learning: Beyond Data Parallelism
27mData Processing on Ray
29mDask-on-Ray: Using Dask and Ray to Analyze Petabytes of Remote Sensing Data