The Challenges Managing a Kubernetes-Based Machine Learning Infrastructure
summary
Panelists explain who uses their platforms, list the concrete stack and tradeoffs, and report operational numbers: on‑prem Bloomberg runs Spark, notebooks, training and model serving; Spotify uses TFX, Ray and GCP; Seldon provides deployment, monitoring and explainability with the V2 inference protocol. They describe three user personas (drag‑drop UI, SDK, super user), multi‑tenancy via namespaces + projects, and migration practices—keeping old APIs, migration docs and a 1,000+ users in support chat escalation process.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 1:02Panelist introductions
- 3:08ML infrastructure evolution
- 5:18ML personas and interfaces
- 8:10Abstraction levels for users
- 10:20Supporting diverse user types
- 13:55CNCF tooling challenges
- 15:20Balancing responsibilities
- 16:02ML Ops culture
- 21:18Handling upgrades
- 24:48Overcoming resistance
- 28:47Guidance for platform engineers
- 32:41Debugging for data scientists
- 37:14Multi-tenancy strategies
- 38:58Closing advice
related talks
18mKeynote: Adapting Kubernetes for Machine Learning Workflows
12mKeynote: Building Enterprise AI/ML Platform on Kubernetes
04mInterview
38mBuilding and Managing a Centralized Kubeflow Platform at Spotify
38mBeyond Kubernetes: Adapting To Spe...
46mMachine Learning with Kubernetes
36mMulti-Cloud Machine Learning Data and Workflow with Kubernetes
32mScaling Kubeflow for Multi-tenancy at Spotify