High-Performance MPI and Deep Learning on OpenPOWER Platform

on YouTube ↗31m

" summary

Provides two commercial packages: a tuned MPI stack and a deep-learning bundle that combine OSU MVAPICH2-GDR with Horovod to run TensorFlow/PyTorch over NVIDIA GPUs and RDMA (InfiniBand/RoCE). Claims a one-command install and prebuilt binaries, shows ~9% higher throughput than NCCL on a DGX test and near-90% scaling efficiency up to about 1,500 GPUs on large systems (includes SUMMIT-scale experiments), and reports the builds run and scale in production.

AI-written summary · about

" chapters (15)

1. 0:00Intro
2. 0:47About us . Bring innovative and efficient and to end solutions, services, support, and training to our customers • Unleash the full potential of your cluster for all your users and applications
3. 2:18Outline
4. 2:53Drivers of Modern HPC Cluster Architectures
5. 5:32Overview of High-Performance Interconnects • High Performance Computing (HPC) has adopted advanced interconnects and protocols
6. 6:19Trends in Microprocessor Technology
7. 7:24MPI + CUDA - Naive Data movement in applications with standard MPI and CUDA interfaces
8. 9:01NVIDIA Collective Communication Library (NCCL) . Collective Communication with a caveat
9. 10:27Why X-ScaleHPC? • How to gain the best possible performance for your application at scale?
10. 11:40CUDA-Aware MPI libraries on OpenPOWER Platform
11. 16:51Deep Learning, Many-cores, and HPC
12. 19:38Scale-up and Scale-out • Scale-up: Intra-node Communication
13. 20:50Data Parallel Deep Learning and MPI Collectives
14. 25:39X-Scale Al: Distributed Training with TensorFlow
15. 28:16Conclusion

" related talks

• 25mHigh-Performance MPI and Deep Learning with Introspection...Dhabaleswar K (DK) Panda, Donglai Dai · OpenPOWER Summit NA 2020
• 45mHigh-Performance and Scalable Middleware for HPC and Deep...Dhabaleswar K (DK) Panda, Donglai Dai · OpenPOWER Summit NA 2021
• 25mHAL: Computer System for Scalable Deep LearningVolodymyr Kindratenko · OpenPOWER Summit NA 2020
• 38mMPI Meets Machine Learning: Unlocking PyTorch distributed for scaling AI workloadsMansi Agarwal · DevConf.IN 2026
• 42mDifferential Software Solutions on HPC and IBM Power ArchitectureAnder Ochoa Gilo · OpenPOWER Summit NA 2021
• 25mHigh Performance Networking for Distributed DL Training in Production K8sNivedita Viswanath · KubeCon + CloudNativeCon North America 2020 Virtual
• 47mDesigning High-Performance Scalable Middleware for HPC, AI, & Data Science in Exascale Systems & ...Hari Subramoni, Dhabaleswar K (DK) Panda · Open Source Summit + Embedded Linux Conference + OSPOCon 2021
• 29mDistributed Deep Learning with HorovodAlex Sergeev · Scale By The Bay 2018