Democratizing Deep Learning at Scale with Horovod
Talkat Open AI + Data Forum @ Open Source Summit North America 2022 · on Tuesday, 2022-06-21· at 16:55· Room 408/409 (Level 4), JW Marriott Austin, 110 E 2nd Street Austin, TX 78701·
summary
Horovod is presented as an open-source framework for scaling deep learning training across many GPUs and machines. The talk covers how Horovod integrates with TensorFlow, PyTorch, and MXNet to enable distributed data-parallel training, plus its techniques for efficient allreduce communication and fault tolerance to democratize large-scale model training.
AI-written summary · about
related talks
29mDistributed Deep Learning with Horovod
27mDistributed Deep Learning with Horovod on Ray
17mAccelerating Collective Communication in Data Parallel Training across Deep Learning...
30mDistributed Deep Neural Network Training using MPI on Python
41mEfficient Data Parallel Distributed Training with Flyte, Spark & Horovod
30mFast and Furious: Practice in Horizon Robotics on Large-scale End-to-end Model Training
29mLarge Scale Distributed Deep Learning with Kubernetes Operators
34mDeep Learning at Scale