Keynote: An Open-Source Stack for AI Compute: Kubernetes + Ray + PyTorch + vLLM
summary
Slides argue a reproducible stack that wires Kubernetes + Ray + PyTorch + vLLM to handle training, multimodal data processing, and serving at scale. Ray provides the distributed compute engine that autos-scales and shards work across GPUs while Kubernetes handles container lifecycle and provisioning. Case claims include companies like Uber and Pinterest adopting it and Instacart moving to 100× more data for training. The talk emphasizes a shift to GPU inference for multimodal data and pragmatic open‑source integrations.
AI-written summary · about
chapters (11)
- 0:00Open-source AI infrastructure
- 1:25AI workloads and Ray's origins
- 3:13Evolving AI workload types
- 4:38Multimodal data processing
- 6:45Tech stack for AI workloads
- 8:31Training and inference frameworks
- 10:39Distributed compute engine
- 11:41Container orchestrator
- 13:03Layered responsibilities and interactions
- 14:29Open-source AI benefits
- 14:49Ray Summit impact
related talks
23mAn Open Source Post-Training Stack: Kubernetes + Ray + PyTorch + vLLM
28mScaling Multimodal Data and Reinforcement Learning with Ray
15mKeynote: From Platforms to AI Factories. Has Kubernetes Solved It?
08mKeynote: Plug in and Scale: Serving LLM Models on Kubernetes Ma...
43mKeynote: Building AI in the Open: How the Cloud Native Community Shapes What Comes Next
32mKeynote: Powering Generative AI with Kubernetes
10mKeynote: Enabling AI with Cloud Native
33mKeynote: Fine-tuning our way towards openness in AI