Evolving the ML Platform Organisation at Netflix: A Case Study
summary
Chronicles how Netflix builds and reorganises ML support: a dedicated Personalization Infra team forms in 2014 to fix feature parity and scaling for hundreds of A/B tests a year, a Machine Learning Infrastructure team launches in late 2017 to speed data scientists into production, and both teams merge recently into a single ML Platform. Key tools and outcomes: Metaflow (now open source) hides data and compute plumbing, algo commons centralises feature encoders, the Java-backed fact store serves shared embeddings for a global userbase of 200 million members.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 2:10ML growth at Netflix
- 5:01Algorithm vs data scientist roles
- 13:13ML infrastructure essentials
- 15:21Origins of personalization infra team
- 17:30Data scientists' infra pain points
- 18:35Forming ML infra team
- 20:16Metaflow and productivity
- 21:47ML teams convergence
- 25:45Merging ML infra teams
- 26:52Key takeaways
