Running Distributed TensorFlow on DC/OS
summary
DC/OS adds a beta tensorflow package that automates distributed TensorFlow deployment: it generates the cluster spec, schedules GPU-aware tasks, restarts failed tasks, injects secrets for cloud storage, and exposes runtime hyperparameter overrides. Demo trains Inception v3 on CIFAR-10 using a 1 master 8 agent cluster on GCE (each agent: four Tesla K80 GPUs, 8 CPUs, 32 GB RAM). Examples run a single-worker config and a 3 GPU workers and 2 parameter servers config; live demo has orchestration hiccups but pre-run TensorBoard data shows the pipeline works.
AI-written summary · about
chapters (42)
- 0:00Overview
- 1:34Introduction to TensorFlow
- 2:05What is Machine Intelligence
- 2:54What is TensorFlow
- 3:43Open Source TensorFlow
- 3:54What is DCOS
- 4:55Highlevel picture of DCOS
- 5:12What are DCOS
- 6:28Deep Learning Overview
- 7:37Training Phase
- 8:50Recap
- 9:32Demo Overview
- 10:21Demo Setup
- 11:03Demo Visualization
- 11:32Summary
- 11:47Demo
- 11:57Cluster Overview
- 12:37DCOS Catalog
- 13:02DCOS Tensorflow Tools
- 13:27DCOS Tensorflow Packages
- 15:29Storage Bucket
- 15:52CPUGPU Allocation
- 16:13Beta Tensorflow
- 16:51Deployment Phase
- 17:36Developer Workflow
- 18:22Moving to Distributed
- 18:36The Real Challenge
- 19:09What DCOS Does
- 19:20Cluster Spec
- 20:07Dealing with failures
- 20:30Manual configuration
- 21:06Highlevel service definition
- 21:42Deployer responsibilities
- 22:16Clean Read
- 22:39DCOS Secrets
- 23:23Runtime Configuration Dictionary
- 26:21Single Framework
- 26:58Meta Framework
- 27:49Flow
- 28:11Special Thanks
- 28:48Questions Links
- 29:35Discussion
related talks
43mDeep learning beyond the learning
34mDemocratizing Machine Learning on Kubernetes [I]
37mBuilding Distributed TensorFlow Using Both GPU and CPU on Kubernetes [I]
38mAll You Need to Know to Build Your GPU Machine Learning Cloud [B]
38mKubernetes for AI Workloads – What Works and What… Doesn’t
37mDistributed Deep Learning on Apache Mesos with GPUs and Gang Scheduling
35mLarge Scale Distributed Deep Learning on Kubernetes Clusters
01h 15mTutorial: Build, Operate, and Use a Multi-Tenant AI Cluster Based Entirely on Open Source