Distributed Deep Learning
summary
Argues that state‑of‑the‑art deep learning now needs GPU clusters and orchestration, cites ImageNet training dropping from weeks to 190 seconds with 1000 GPUs, and shows an open source stack called Hops platfrom that integrates a versioned feature store, Spark/PySpark preprocessing, distributed TensorFlow with ring AllReduce, Kubernetes model serving, and NVMe‑backed hopsFS optimizations that claim up to 66x smaller‑file throughput improvements and reuse of Uber‑scale feature libraries (Uber reports ~10,000 features).
AI-written summary · about
chapters (16)
- 0:00Start
- 4:26Model Training Just One Piece of the Puzzle
- 7:56ML Pipelines in Hopsworks
- 14:00A Fully Distributed Deep Learning Pipeline
- 14:55Hopsworks Machine Learning Pipeline
- 16:31Data Problem
- 17:45Hopsworks' Feature Store
- 22:14Building Blocks of a Feature Store
- 23:01Feature Storage
- 23:23Feature Metadata
- 28:13Distributed Filesystem for Coordination
- 29:38NVMe Disks - Game Changer
- 32:24Faster Experimentation
- 35:03Data Parallel (Distributed) Training
- 36:29Data Parallel Training
- 41:50Model Serving
related talks
34mDemocratizing Machine Learning on Kubernetes [I]
34mDeep Learning at Scale
37mDistributed Deep Learning on Apache Mesos with GPUs and Gang Scheduling
43mDeep learning beyond the learning
16m[Opening Keynote] Making Distributed Computing Easy
22mBringing Deep Learning into the Big Data Analytics
41mScaleable Distributed Deep Learning
36mAccelerating High-Performance Machine Learning at Scale i...