Reducing Operational Barriers to Model Training
summary
Hardware barriers fall but shipping experiments still stalls teams: initial workflow copies files via SCP and TMUX windows, then shifts to building a local image, pushing to a registry and running containers, and finally to Kubernetes on Amazon EKS to run many hyperparameter trials in parallel (six models cuts six days to one). The project builds SigOpt Orchestrate, measures throughput and productivity gains, and exposes a simple CLI that wraps infra APIs with Docker, Kubernetes (Amazon EKS), SigOpt Orchestrate, logs/status CLI.
AI-written summary · about
chapters (8)
- 0:00Intro
- 3:31Operational Barriers
- 6:56Can't Train Model on Laptop
- 7:48Containerize!
- 8:45Start Training in Parallel
- 9:45Kubernetes!
- 11:49View Progress and Debug
- 13:05Build a CLI!
related talks
29mTeam Teardown: MLOps in the Cloud for IoT
24mDemo of Kubernetes at the Edge for AI Model Deployments
43mProblems and Solutions with AI Model Deployments in k8s
12mLightning Talk: Benchmarking Your Distributed ML Training on the K8s Platform
27mHarnessing Kubernetes for Scalable AI/ML Workloads
19mNo Token Budget: Agentic Software Factories on Kubernetes
22mEmbedded Software Development Efficiency Improvement with Docker and Kubernetes
18mBaby Sitting Your AI Models in Production? STOP!