Writing robust, readable, and maintainable concurrent programs in Python
summary
Compares Python concurrency models with code and timings: threading suffers from the global interpreter lock so CPU-bound threads are slower and produce incorrect layer ordering; multiprocessing spawns separate interpreters and using a process pool executor cuts an 80s sequential workload to ~21s for four workers; combining per-process async fan-out/fan-in (threads for I/O) shaves another ~5s to ~16s; recommends no shared state, idempotent updates, concurrent.futures for returns, and producer‑consumer queues for pipelining.
AI-written summary · about
chapters (11)
- 0:00Intro
- 2:42Making of kue lapis sagu
- 6:02Multithreading
- 10:40Multiprocessing
- 12:44Solution 1: Use lock?
- 14:00Solution 2: Always design for concurrency
- 18:34Idempotent state
- 21:04Restaurant queue
- 23:09Embarrassingly Parallel
- 26:54Producer Consumer Queue
- 29:30Summary
related talks
31mPython Multithreading and Multiprocessing: Concurrency and Parallelism
02h 57mPython Concurrency: from beginner to pro
36mConcurrency and Parallelism with Python for Humans
10mRedlining Your CPU for Fun and Profit: Threading and Multiprocessing in Python
30mParallel computing in Python: Current state and recent advances
32mPython's Parallel Programming Possibilities - 4 levels of concurrency
30mSpeed Up Your Data Processing
30mPython Serialization and the Hallows of Parallel Computing