Working with multiple hands: Parallel and Distributed Programming
summary
Demonstrates using mpi4py with mpiexec to run multiple Python processes (example: five-process “Hello World” and a 4-worker numerical integration) and shows concrete calls: mpi4py, mpiexec, comm.reduce. Claims near-C speed with pickle-based communication, and that simple problems often double or triple performance on one machine by using processes not threads. Covers send/recv semantics (blocking), broadcasts vs many sends, machinefile for distributed nodes, and practical pitfalls like hard debugging and communication vs computation tradeoffs.
AI-written summary · about
chapters (10)
- 0:00Introduction and speakers
- 0:48Concurrent vs parallel programming
- 4:22Message Passing Interface overview
- 6:36MPI in Python
- 8:42Hello World with MPI
- 13:36Area under curve example
- 17:30Collective operations in MPI
- 19:56Distributed programming with MPI
- 20:57Best practices for MPI programs
- 22:23MPI trivia and Q&A
related talks
10mMPI Meets Python: How to Use MPI in the Python Universe
56mImplementing distributed applications using ZeroMQ, Python and other bad guys...
38mMPI Meets Machine Learning: Unlocking PyTorch distributed for scaling AI workloads
23mA Haskell Auto-Parallelizer for Distributed Computing
30mThree Goals in Parallel Graph Computations: High Performance, High Productivity, and Reduced Comm...
19mPurrrfectly parallel, purrrfectly distributed
12mMonarch: An API To Your Supercomputer
11mPyTorch Distributed RPC