Case Study: Developing and Scaling Kubeflow’s Web Apps
summary
Kubeflow maintains at least five web apps (Notebooks, PVCs, Tensorboards, Models, AutoML experiments) that perform CRUD on K8s objects and scale by adding pods; team isolation uses namespace isolation with authentication via Istio/Dex and authorization via SubjectAccessReviews so backends act on users’ behalf. Auto-refresh uses exponential polling (1,2,4… up to 32s) with a planned optimization to HEAD on resourceVersion plus Kubernetes API chunking; evaluated WebSockets/SSE. The AutoML stack uses a Kubernetes-native project called k-dip/cadib and the UI surfaces experiment, trial, and model metrics; project shows production-ready dashboards and reports 12k commits and ~22k GitHub stars.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 7:30What is Ultima
- 11:39Central Dashboard
- 12:31Notebooks
- 13:23Pipelines
- 14:34Validity Experiment
- 14:47Search Space
- 15:09Matrix Collector
- 16:01Experiments
- 17:46Models UI
- 17:59Models Metrics
- 19:02Community
related talks
34mKubeflow: Multi-Tenant, Self-Serve, Accelerated Platform for Practitioners
31mKubeflow - Data Science on Steroids
42mKubeflow From the End User’s Perspective: The Good, The Bad, and The Ugly
21mIntroducing Kubeflow (w. Special Guests Tensorflow and Apache Spark)
14mKubeflow: Where ML Meets Kubernetes
46mDeploying scalable Machine Learning models with Kubernetes
54mEconomics and Best Practices of Running AI/ML Workloads on Kubernetes
36mIntroducing Kubeflow (w. Special Guests Tensorflow and Apache Spark)