Ray Summit 2026
2026-08-24..2026-08-26
San Francisco, United States
[Ray Summit 2020, Ray Summit 2021, Ray Summit 2024, Ray Summit 2026]
as the organisers put it
Join Ray Summit in San Francisco, Aug 24–26, for technical talks on foundation model training, multimodal AI, RL, and other AI in production systems.
| title | author | length |
|---|---|---|
| Diffusion Large Language Models | Volodymyr Kuleshov | 40m |
Inception builds a diffusion LLM called Mercury 2 that claims to be the fastest model on the Artificial Analysis index and to deliver 3–10x latency speedup… | ||
| How the Data Center Shapes the Workload | Oscar Savolainen | 12m |
Describes a fleet manager implemented as a state machine from power‑on and multi‑day burn‑in to Kubernetes admission that filters silent stragglers before… | ||
| Inside BMW's AI Gateway: Self-Hosting Cost-Efficient AI at Enterprise Scale | Thomas Riedl | 28m |
BMW’s AI Gateway moves from hyperscaler APIs to self-hosting, deploying Hugging Face models with KubeRay with vLLM on AWS EKS and vLLM Omni for diffusion… | ||
| Inside Lila's AI Research Platform | Priunsh Syen, Tyler Titsworth | 33m |
Lila Sciences runs a self‑service, cloud‑native AI platform that claims to handle multicluster, multicloud, multiregion workloads and to shrink ideation time… | ||
| Leveraging Agent Skills for Ray Workloads | Aydin Abiar, Goku Mohandas, Kunling Geng | 31m |
Anyscale ships an agent-native CLI plus SDK and MCP hooks that let coding agents (cloud code, cursor, CodeX, GitHub Copilot) run locally to spin up Ray… | ||
| Multi-Region, Environment Isolated Ray Environments in the Notion Way | Nathan Louie, Yunhao Qing, Preeti Gondi | 21m |
Notion rearchitects AnyScale deployment to enforce true region-and-environment isolation by creating 8 clouds (one per region per environment), moving all jobs… | ||
| Adapting Ray to Scale RL Training for Nemotron | Wenwen Gao, Saurabh Mishra | 26m |
NVIDIA adapts Ray to run Nemotron RL at 3K+ GPU scale by changing Ray configs, reducing actor count 42%, and moving heavy GCS work into separate IO contexts… | ||
| LeRobot Dataset v3.0: Scaling Reads with Ray | Artur Niederfahrenhorst | 14m |
LeRobot v3 data causes heavy redundant MP4 decodes because episodes span multiple video files and files contain many episodes; Ray Data implements read… | ||
| From OOMs to 2x Throughput: Scaling Multimodal Data Curation on Ray | Anindya Saha | 31m |
Shows a concrete pipeline that extracts camera, lidar, and vehicle state from terabytes of proprietary logs using Ray Data streaming plus Ray Core task… | ||
| Agents and Robots: The Human Effort Putting AI Into Production | Oscar Evans | 08m |
Encord ingests multimodal logs (LiDAR, depth, teleop, video) from cloud storage, searches at scale for long‑tail events, and applies model‑led prelabeling… | ||
| Async Inference at Netflix: Batch and Near-Real-Time Serving | Wenbing Bai, Naveen Somasundaram | 37m |
Netflix runs a unified async inference platform that executes the same pipeline code in two modes: ephemeral Ray Data jobs for high-throughput batch and warm… | ||
| Building a Virtual Biology Platform on Ray | Aleksandar Djuric | 31m |
Recursion runs a virtual biology platform on Ray and Anyscale over a 60 PB dataset with 40 PB proprietary, claiming to cut cloud spend while improving… | ||
| Building Self-Serve, Multimodal Medical Image Pipelines with Ray | Deepak Sharma, Patrick Violette | 29m |
Heartflow builds PAL: a self-serve pipeline that converts S3 blob scans into Iceberg tables via a bronze/silver/gold medallion, using Ray Data for distributed… | ||
| Cascade: Write-Once Run-Anywhere Inference on Ray | Eagle Lo, Hao Yuan, Freeman Mak | 24m |
Cascade compiles DSL-defined DAGs into Ray Serve, Ray Data or Spark runtimes so one logical ML pipeline runs unchanged for HTTP real‑time serving and… | ||
| Custom Pipeline Parallelism for Frontier-Scale Model Serving | Abin Shahab, Michael Lee | 22m |
vLLM’s compiled-DAG deadlocks beyond two pipeline stages, so the team replaces it with a custom Ray executor that uses actor mailboxes as the scheduler and… | ||
| From Agents to Action: Scaling Agentic AI with Ray on Intel Xeon | Srikanthan Srinarayan | 17m |
Describes deploying agentic pipelines that push tool execution and retrieval onto Intel Xeon 6 CPUs while using Ray for distributed orchestration and parallel… | ||
| From Reservation to 1,600 GPUs: A Petabyte-Scale Ray Data Pipeline | Xinyu Zhang, Jeff Braunstein | 14m |
Anyscale and CoreWeave deploy a pipeline that turns a fresh GPU reservation into production in under 24 hours, moving from Ray Core to Ray Data and streaming… | ||
| GammaLake: High-Performance Open-Source Feature Storage for Timeseries | Todd Gaugler | 30m |
GammaLake is an open-source Point72 project that layers Delta tables with structured tall-and-skinny representations (global sortable index like… | ||
| Miles: Enterprise-Facing Agentic RL Framework | Mao Cheng, Xinyu Zhang | 29m |
Miles is an open-source PyTorch + Megatron + Ray stack that runs fully async agentic RL at frontier scale: a GM‑5.2 reference run hits ~263s median per step… | ||
| Lessons from Unifying Simulation and RL Training | Neil Wadhvana, Yashovardhan Chaturvedi | 27m |
Torc unifies object-sim and end-to-end neural-rendering RL on Ray and AnyScale, swapping GPU-drive sims for a CPU-vectorized stack to hit 2 billion steps in 40… | ||
| How Open-Source vLLM Topped the Artificial Analysis Leaderboard | Debarshi Raha | 13m |
Open-source vLLM plus DigitalOcean infra delivers concrete wins: B100 GPUs with NVB4 quantization and tensor sharding cut memory and speed bottlenecks, kernel… | ||
| How Exa Indexes the Entire Web: Ray Data and Lance at Billion-Document Scale | Lei Xu, Hubert Yuan | 33m |
Exa builds a petabyte-scale derived-data engine that stores documents in Lance’s two-dimensional disk layout so columns grow independently and new column files… | ||
| Evolving and Scaling Ray for the Next Generation of AI Workloads | Mengjin Yan, Andrew Sy Kim | 27m |
Ray Core now pushes control‑plane and driver limits by offloading high‑volume events, cutting RPCs and lock contention to deliver 1.6x task throughput and… | ||
| Heterogeneous Pipelines as the Unlock for ML at Scale | Tim Cheng, Hareesh Kolluru | 23m |
Motive rebuilds its ML eval pipeline on Ray so each stage runs on a right-sized tier: CPU-heavy fetch/decode on wide cheap instances, GPU inference on a… | ||
| Full-Stack Observability in Ray: Infrastructure Signals in Ray Dashboard | Richa Banker | 15m |
Debugging distributed ML jobs on Kubernetes is difficult because of the visibility gap between failures at the infrastructure layer and the framework… | ||
| Koba: Disaggregated Data Loading for Multimodal Training at Luma AI | Minghao Xie, Haoran Cai | 32m |
Koba is a PyTorch-native, Lens-integrated multimodal data loader and Kojo is a Ray/Kubernetes-backed disaggregated preprocessing pool that offloads… | ||
| Maximizing GB200 and GB300 Performance with Domain-Aware Scheduling | Josh Lee, Connor Pedersen | 24m |
Ray adds a placement-group field called topology_strategy that enforces rack-level locality (strict_pack_on_racks) so bundles bin-pack into a single… | ||
| One Cluster, End-to-End | Shreyas Adiyodi, Anoop Saha | 13m |
Demonstrates collapsing data prep, distributed training, and serving onto one Amazon EKS cluster using SageMaker HyperPod, running heterogeneous CPU/GPU Ray… | ||
| Operating a Production-Scale Physical AI Data Platform with Ray | Hyeonggyu Kim | 35m |
Hyundai runs Ray as the foundation of a physical AI data platform processing petabyte-scale sensor data for autonomous driving, automating model-based curation… | ||
| Performant Training for All Workstreams | Krishna Toshniwal | 28m |
Latitude measures end-to-end wall-clock throughput rather than GPU utilization and instruments Ray Data + Ray actors to break training into load, transfer, and… | ||
| Petabyte-Scale Video Curation on CoreWeave | Jeff Braunstein | 26m |
Pipeline curates 600 TB of video on CoreWeave using Anyscale managed Ray + Ray Data on CKS, decoding with FFmpeg, sampling frames, and running VLLM inference… | ||
| Platform Orchestration: Anyscale + Ray on Kubernetes | Adhip Gupta, Elizabeth Hu | 17m |
Anyscale on Azure integrates into AKS running in the customer's subscription with ARM-native provisioning, Entra ID workload identity, and data that never… | ||
| Ray Across the Stack Fine Tuning Qwen and Serving 3D Generation at Autodesk with AWS Trainium and In | Cong Luong, Victor Oliveira Antonino | 30m |
Autodesk fine-tunes Qwen3 on AWS Trainium, using Ray Train and Hugging Face Optimum Neuron to shard weights across 32 NeuronCores with TP32 sharding and runs… | ||
| Ray and MAI-Thinking-1 | Shaowei Su | 28m |
Microsoft AI uses Ray as the shared compute layer to build MAI-Thinking-1, running across Kubernetes and Slurm with multi‑cluster setups. They implement relay… | ||
| Ray Data on GPU - Breaking Preprocessing Bottleneck for Model Training | Lee Yang, Pratik Gujjar | 29m |
Stripe moves large tabular pipelines (1.9T annual volume, 6–10 TB / ~2B rows) onto GPUs by adding cuDF as a Ray Data batch format, using Ray map_batches plus a… | ||
| Ray Data: What's New - GPU Shuffle, Unity Catalog, DataSourceV2 | Justin Hsu, Goutam Venkatramanan | 36m |
Ray Data 2.53–2.58 adds an actor-only model that replaces mixed tasks to enforce per-actor buffer limits, reducing cross-node spilling and improving pipeline… | ||
| Ray History Server: Observability for Ephemeral K8s Clusters | JiaWei Jiang, Aaron Liang, Hanju Chen | 17m |
KubeRay adds a Ray History Server that keeps observability after ephemeral clusters die by writing logs and events to object storage and replaying them into… | ||
| Ray in Production: Engineering Lessons from Spotify's Hendrix Framework | Aamir Anwar | 32m |
Spotify’s Hendrix platform runs Ray on Kubernetes to provision on-demand GPU infrastructure, orchestrate >1 million GPU hours/month, and serve >10,000 batch… | ||
| Ray Train on Anyscale: Observability and Reliability at Scale | Jason Li, Mark Towers | 21m |
Ray Train on Anyscale adds data ingest observability (Ray Data) to show that >30% of an average batch cycle can be blocked by a slow collate stage, pinning the… | ||
| Right Resource, Right Job: Heterogeneous Ray Clusters on Amazon EKS | Robert Macy, Viraj Padte | 30m |
Robinhood moves a homogeneous RayCluster on Amazon EKS to a heterogeneous setup using KubeRay, Carpenter node pools, Ray Data and Ray Train, and custom… | ||
| Reward Stress Tests for Agentic Post-Training with Ray | Radhika Gaonkar | 33m |
Describes a Ray-backed workflow that stress-tests reward pipelines before scaling RL: synthetic injection of ~4,600 mechanical mutations yields ~1,000… | ||
| Satellite Imagery to Machine-Readable Intelligence | Milos Colic | 39m |
Claims a pipeline that turns multi-band, temporal Sentinel data into one embedding per pixel using the open TERA model, stores and parallel-writes chunked… | ||
| Scaling DSpark Training Using vLLM, Speculators and Mooncake | Helen Zhao | 17m |
A reproducible pipeline trains DSpark drafters by streaming hidden states from a GLM5.2 verifier into a distributed trainer, using vLLM extractors, Mooncake as… | ||
| Scaling Frontier AI with Ray on Kubernetes | Andrew Sy Kim, Jui-An Huang, Hanju Chen | 19m |
KubeRay is now the production Kubernetes operator for Ray and shows concrete wins: a history server that persists Ray logs and replays dashboards, native token… | ||
| Scaling LLM Workloads on TPUs with Ray: From Pre-Training to Inference | Ryan O'Leary, Lehui Liu | 28m |
Ray on GKE implements topology-aware scheduling that atomically reserves contiguous TPU slices (examples: 4-node 4x4 or 2x2x2 topologies) to avoid XLA… | ||
| Scaling Machine Learning at Discord: Our Journey with Ray and Anyscale | Serrana Aguirregaray | 30m |
Discord moves from single-node XGBoost to a distributed multitask neural ranker on Ray and Anyscale, shipping a production model that trains on 2 billion raw… | ||
| Scaling Massive Transformer Training with KubeRay | Mauilin Patel, Raja Chawat, Yifei Wang | 16m |
Capital One runs KubeRay on Kubernetes to stream 3.5TB (≈2 billion records) from S3 into GPUs and consolidates Ray Data, Ray Train and Ray Tune into one… | ||
| Scaling PyTorch Training with Ray Data on Uber's Michelangelo AI Platform | Ke Chen, Arushi Arora | 33m |
Uber’s Michelangelo migrates feature transforms and dataloaders to migrated from Spark to native PyTorch on Ray Data and replaced Petastorm with Ray Data… | ||
| Scaling RL with Ray, Kubernetes, and TPUs | Poonam Lamba, Nisha Mariam Johnson | 34m |
Reference architecture using Ray + GKE + TPUs with concrete numbers and tools: Ray adds sub-second cold starts via native gVisor sandboxes and promises… | ||
| Scaling the Ray Serve Controller to Thousands of Replicas | Abrar Sheikh | 23m |
Systematic scalability experiments push a single Serve controller to 2,048 replicas and break down where time goes: heavy cost comes from Pydantic… | ||
| From Self-Managed Ray to Anyscale Services: Migrating ML Serving | Aman Choudhary, Akshit Trehan | 16m |
Coinbase migrates production serving from self‑managed Ray Serve on EKS to Anyscale managed Ray, moving 50+ models and 1 billion inferences per day into… | ||
| From Benchmark #1 to Production | Ryan Wang | 16m |
Nebius claims production-ready inference wins: a Kernel Design Agent that auto-generates and verifies kernels yields an ~8x speedup on a deepsea… | ||
| Anyscale on Azure: Build, Train, and Serve AI in Your Own Tenant | Daniel Arrizza, Bob Mital | 14m |
Anyscale on Azure is an Azure‑native, co‑engineered platform that runs Ray on AKS with the control plane in Azure and the runtime inside the customer’s AKS… | ||
| Closing the Train-Inference Divide for Asynchronous RL at Scale | Dominic Yurk | 14m |
Implements faithful replay of router decisions and per-token top-K/top-p support in SkyRL, capturing rollout metadata and reconstructing it exactly on trainer… | ||
| Cost-Effective RL Post-Training with Ray and AWS Trainium | Chelsie Zhang, Zhenkun Cai, Artur Niederfahrenhorst | 23m |
Port of a Ray-based RL post-training pipeline to AWS Trainium uses AWS Neuron and PyTorch so existing Ray recipes run unchanged except for device="neuron"… | ||
| Serving a Frontier Open Model with Ray & vLLM | Eyal Lantzman | 28m |
Deploys DeepSeek-V4-Flash on three B200 GPUs in Kubernetes using Ray and vLLM and shows a simpler tensor+expert setup hitting ~2,200 aggregated tokens/sec with… | ||
| Serving Fast and Smart: Getting the Most Out of Ray Serve LLM on GKE | Seiji Eicher, Spencer Peterson | 18m |
Ray Serve LLM on GKE replaces the Python proxy with HAProxy and splits control and data planes via an ingress request router so streaming tokens go directly to… | ||
| Serving Frontier MoE Models at the Lowest Token Cost | Amr Elmeleegy | 14m |
A 72‑GPU rack called GB300 NVL72 is built with 18 trays and nine NVLink switches to give each GPU 1,800 GB/s and an all‑to‑all fabric of 130 TB/s, eliminating… | ||
| KV Cache and Long-Context Inference: Solutions for Inference at Scale | Vaughn Stewart | 12m |
Offloading KV cache to VAST Data’s flash layer with NVIDIA Dynamo and BlueField‑4/CMX shifts recompute to reads, tested on H100/H8 hardware with 128k context… | ||
| Serving LLMs for Agents at Scale with Ray + vLLM | Ankur Mohan, Dhayaneshwar Balusamy | 28m |
Self-hosted Ray + vLLM platform reduces end-to-end latency from ~2,400 ms to 275 ms and reports about 10x performance and 12x cost versus third‑party hosts… | ||
| Tailor-Made Inference at Scale | Ansh Pratap Singh | 13m |
A configurable inference engine breaks inference into swappable atomic components and an agent-driven search composes stacks from vLLM and TensorRT-LLM… | ||
| Simulating the Physical World at Scale: Humanoid Locomotion with Ray | Amir Zadeh | 13m |
Lambda uses Ray and NVIDIA Isaac Sim to scale Unitree G1 gait fine‑tuning across a two‑node Lambda GPU cluster (16 GPUs) running 512 environments per GPU for a… | ||
| SkyRL: Democratizing RL Training for Researchers | Sumanth Hegde, Eric Tang | 36m |
SkyRL presents a modular RL stack that ties Megatron training, vLLM serving, and Ray orchestration into fully async RL pipelines that run 350B+ MoE and… | ||
| The Synthetic Data Flywheel | Eric Wang, Gaurav Arora | 34m |
Pinterest builds a Ray Data pipeline that turns seed queries into millions of synthetic preference pairs by streaming 256 upfront partitions through GPU… | ||
| Towards Clinically Grounded LLMs | Anne Clarke, Akele Reed | 30m |
Fine-tuning with LoRA adapters on heavily de-identified real therapist turns shifts baseline LLM behavior toward human therapists across understanding, guided… | ||
| Training Frontier Knowledge Work Agents: A 397B Open Recipe with SkyRL | Edward Hu | 39m |
Mercor post-trains knowledge‑work agents with SkyRL and Harbor’s Archipelago harness, scaling from 35B to a 397B model on 20 H200 nodes via Ray, and publishes… | ||
| Transforming Grab’s AI Platform with Ray | Yvonne Wong, Chongyu Zhou, Abhinav Rai | 30m |
Grab consolidates offline training, simulator parameter search, and RL on one Ray layer, using Ray Train, Data, Tune, and Core plus Optuna to automate searches… | ||
| Trillion-Token Pretraining | Hanna van der Vlis | 33m |
Pretraining a payments foundation model on trillions of tokens builds a hierarchical transformer that first encodes per-transaction features then attends… | ||
| Vertical Mobility: Inference from MVP to Trillion Parameters | Sitanshu Gupta | 12m |
A multi‑tier inference platform is shown that scales from pay‑per‑token experiments to single‑tenant GPU leases and self‑managed clusters, implemented as… | ||
| Video Action Models | Elvis Nava | 32m |
Argues that pre-training a generative video backbone and adding an action decoder yields much higher sample efficiency for robot manipulation — reported as 10x… | ||
| Where’s My Training Job? Ray & GPU Observability at Datadog | Marina Petzel, Ashley Chu | 12m |
Datadog builds a unified job-level view that preserves Ray job identity across cluster lifetimes, ties each rank to process, host, GPU, traces, and topology… | ||
| Why Current AI Cannot Truly See the World | Andrew Dai | 32m |
Claims that frontier multimodal models systematically fail on grounded visual reasoning, with concrete examples: a railing image has 37 rails answered 2, a… | ||
| nothing matches | ||