Parallelism the Old Way: Using MPI in Python with mpi4py
summary
Demonstrates using MPI4Py for real Python parallelism: shows blocking send/recv, how small messages get copied but larger ones hit a ~32KB buffering cutoff and can deadlock, and how non-blocking send/receive with an explicit byte-buffer avoids that. Builds a task farm and grid-decomposition Game of Life: task-farm tests claim ~5× speedup on 8 processes (3.1× on 4), and a 20-core Mac Studio run drops a test suite to about 10 seconds.
AI-written summary · about
chapters (13)
- 0:00Introduction to MPI
- 4:46Understanding parallelism
- 5:03Architectures and models
- 7:25Parallel computing types
- 12:57MIMD and MPI basics
- 16:18Distributed memory systems
- 20:00MPI history and concepts
- 25:40Programming with MPI
- 30:16Blocking communication
- 43:53Deadlock and non-blocking
- 59:15Advanced MPI techniques
- 1:10:30Data and task parallelism
- 1:20:52Grid decomposition example
related talks
01h 01mParallel Python Analyzing Large Data Sets
01h 07mGeneric Parallel Programming
01h 06mBodo: Automatic HPC Performance and Scaling for Data Processing in Python
56mData Analysis in Parallel
59mPython Meets Heterogeneous Computing
01h 22mUsing Dask for Parallel Computing in Python
38mMPI Meets Machine Learning: Unlocking PyTorch distributed for scaling AI workloads
47mDesigning High-Performance Scalable Middleware for HPC, AI, & Data Science in Exascale Systems & ...