Ray History Server: Observability for Ephemeral K8s Clusters
summary
KubeRay adds a Ray History Server that keeps observability after ephemeral clusters die by writing logs and events to object storage and replaying them into the Ray dashboard. It deploys lightweight sidecars to read shared tmp volumes and accept HTTP-batched events, handles graceful shutdowns, session transitions, and collector restarts, and uploads rotated JSONL files with backpressure. A stateless replay server reads GCS (or other) buckets, uses on-demand loading plus an LRU bounded cache to serve many sessions without idle compute; alpha → beta and 100+ PRs.
AI-written summary · about
chapters (8)
- 0:00Introduction and overview
- 1:11Observability problem and solution
- 2:58What is the history server
- 4:01Live demo of history server
- 7:19Collector configuration details
- 12:15History server architecture
- 15:51Roadmap and future plans
- 16:56Q&A and resources
related talks
28mRay on Public-Cloud k8s - experiments, lessons & best practices
24mSkyRay: Seamlessly Extending KubeRay to Multi-Cluster Multi-Cloud Operation
39mSailing Ray Workloads with KubeRay and Kueue in Kubernetes
32mAn Overview of CloudKitchens's Ray-Powered ML Platform
22mDebugging and Observability for Distributed Ray Applications
26mIntroduction to Distributed ML Workloads with Ray on Kubernetes
31mNext-Generation Data Science Workflows Using Ray
35mScaling Ray Train to 10K Kubernetes Nodes on GKE