Efficient Model Exploring and Continuous Delivery With Polyaxon + Kubeflow
summary
Mercari describes using Polyaxon for scalable hyperparameter search and Kubeflow Pipelines for automated continuous training and delivery, with Spinnaker triggering deploys after Kubeflow builds and pushes new model images. The platform runs 175 projects and reports about 87,000 experiments from fewer than 10 developers, provides a custom Kubeflow component that submits Polyaxon jobs, CI compiles/uploads pipeline definitions to dev/prod clusters, and the end-to-end flow successfully moves models from exploration to production automatically.
AI-written summary · about
chapters (14)
- 0:00Intro
- 0:20What is Mercari?
- 1:12Machine Learning at Mercari US
- 3:12ML Development Lifecycle
- 4:43ML Project Lifecycle at Mercari US
- 6:09What is Polyaxon?
- 7:25How to run a job on Polyaxon
- 9:27What is Kubeflow Pipelines?
- 10:48Kubeflow Pipelines at Mercari US
- 12:28What we built to accelerate iterations
- 14:02Monorepo for Kubeflow Pipelines
- 16:10"Project" Manifest for KFP and Polyaxon
- 17:14Polyaxon Kubeflow Pipelines Component
- 18:41Takeaways
related talks
27mKubeflow for enabling AI Powered Drug Discovery and development...
38mAutomating Machine Learning and Deep Learning Workflows
01h 18mScaling and reproducing deep learning on Kubernetes with Polyaxon
54mScaling Machine Learning Pipelines in Cloud
14mKubeflow: Where ML Meets Kubernetes
31mKubeflow 2026 State of the Union: Orchestrating the Full ML Lifecycle
29mML Engineering Kubeflow
34mModern Machine Learning flow with Quilt and Polyaxon