Airbnb's LLM Evolution: Fine-Tuning with Ray
summary
Airbnb migrates its model layer onto Ray and builds a one-job-per-cluster pipeline that combines Llama Factory for unified open‑source models, Kueue for preemptive and guaranteed GPU queues, and Ray Train for distributed fine‑tuning. A text‑summarization pipeline uses human labeled baselines, runs SFT then DPO, and reports that an 11B T5 variant surpasses prior human and GPT‑4 baselines on the product metric. The platform emphasizes periodic realignment, batch inference integration, and autoscaling to raise GPU utilization.
AI-written summary · about
chapters (14)
- 0:00Introduction and agenda
- 1:03Legacy infrastructure
- 2:32Ray platform architecture
- 3:51Functional infrastructure layers
- 6:31Scaling and development challenges
- 7:51Open source framework adoption
- 10:11Workflow standardization with GLOW
- 11:55Case study overview
- 12:57Fine-tuning strategy
- 15:40Reinforcement learning pipeline
- 17:27Model alignment and evaluation
- 19:05Direct preference optimization
- 20:51Results and takeaways
- 22:17Q&A session
related talks
21mEnd-to-End AutoML with Ludwig on Ray
44mPanel: ML Platform on Ray
21mHow Ray and Anyscale Make it Easy to do Massive-scale ML on Aerial Imagery
22mRay, a Unified Distributed Framework for the Modern AI Stack
35mOperating a Production-Scale Physical AI Data Platform with Ray
28mScaling Multimodal Data and Reinforcement Learning with Ray
28mScaling AI & Machine Learning Workloads With Ray on AWS, Kubernetes, & BERT
23mAn Open Source Post-Training Stack: Kubernetes + Ray + PyTorch + vLLM