Running Machine Learning Workloads on a Service Mesh
summary
Adds Istio to a JupyterHub Kubernetes deployment, rewrites the proxy to fix header routing, introduces per-user Kubernetes Services so hub can probe notebooks, then removes the bundled proxy and replaces it with a lightweight JupyterHub proxy API behind an Istio gateway to scale. Achieves mutual TLS in strict mode, uses Prometheus metrics for mTLS compliance, and reports that the redesigned system handles larger user counts and operational telemetry successfully.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 0:44Agenda
- 1:36Background
- 3:38What is Machine Learning
- 8:54The Problem
- 21:19Compliance Audit
- 22:39Deployments
- 24:58Conclusion
related talks
29mHow to Improve Your Kubernetes Experience with Service Mesh and MLOps
23mManage Multi-tenant ML Workloads Using Istio
18mLessons from Scaling AI Powered Translation Services Using Istio
35mCase Study: AI-as-a-Service on Kubernetes at Scale and In Production
42mThe Challenges Managing a Kubernetes-Based Machine Learning Infrastructure
36mMulti-Cloud Machine Learning Data and Workflow with Kubernetes
46mDeploying scalable Machine Learning models with Kubernetes
28mMachine Learning on Kubernetes at Shell: A Kubeflow Journey