Deep learning beyond the learning
summary
Pipeline covers training to production with concrete choices: use Kubeflow or cloud ML engines for notebooks and orchestration, store hundreds of model versions in HDFS/GFS plus a metadata DB, serve with TensorFlow Serving or a rendezvous pattern feeding multiple models, and shrink mobile models with TF Lite. Expect iterative runs (hundreds of hyperparameter trials), heavy data work (≈50% of effort), GPU/TPU use, automated version policy for live swapovers, and TF Debug/tensorboard for profiling and early stopping.
AI-written summary · about
chapters (19)
- 0:00Intro
- 0:53Deep Learning: The Promise
- 4:14Deep Learning: The Challenges
- 7:40Deep Learning Pipeline
- 10:20Training Challenges
- 12:04Data Preparation
- 13:21Input Data Management
- 18:18Alternatives
- 20:05Data Analytics Ecosystem
- 23:53Resource Isolation and Allocation
- 25:12Datacenter
- 28:41MESOS ARCHITECTURE
- 29:58Model Management
- 32:57Model Serving
- 34:39Rendezvous Architecture
- 36:02Monitoring
- 36:48Debugging
- 38:24Profiling
- 38:50Platforms
related talks
39mOperating Deep Learning Pipelines Anywhere Using Kubeflow
43mDeep learning beyond the learning
46mAccelerating Machine Learning DevOps with Kubeflow
54mScaling Machine Learning Pipelines in Cloud
46mDeploying scalable Machine Learning models with Kubernetes
29mA Scalable Platform for Training and Inference Using Kubeflow
21mMastering the DevOps Machine Learning Pipeline
25mContinuous ML Applications in Production