Exploring Hinge's ML Platform Evolution with Ray
summary
Hinge migrates from EMR Spark to Ray on Kubernetes to shorten time‑to‑production, running Ray clusters via Argo CD for GitOps with GitHub Actions building images and simple cluster configs that let users scale CPU/GPU replicas; observability uses Prometheus/Grafana and experiments use Weights & Biases. Last‑mile feature processing shifts from Spark to Ray reading Iceberg tables (an Iceberg connector is contributed back) and serving uses Ray Serve. A 20‑person ML org and a 10M MAU product reports a ~3–4 month adoption curve for this stack.
AI-written summary · about
chapters (7)
- 0:00Introduction and context
- 2:01ML landscape at Hinge
- 4:35Platform architecture
- 8:11Ray cluster management
- 13:08Data processing pipelines
- 14:32Future platform goals
- 17:03Q&A session
related talks
44mPanel: ML Platform on Ray
26mA Growing Ecosystem of Scalable ML Libraries on Ray
26mIntroduction to Distributed ML Workloads with Ray on Kubernetes
32mRay in Production: Engineering Lessons from Spotify's Hendrix Framework
16mFrom Self-Managed Ray to Anyscale Services: Migrating ML Serving
31mNext-Generation Data Science Workflows Using Ray
35mOperating a Production-Scale Physical AI Data Platform with Ray
29mAdopting Ray @ LinkedIn