Trying No GIL on Scientific Programming
summary
Claim: nogil fork on Python 3.9 can remove the GIL and speed CPU‑bound scientific code. Experiment uses scikit‑learn, NumPy and SciPy examples, C extensions where present, and ran each test 50 times on a dual‑core MacBook. Results are mixed: most benchmarks show no significant speedup, linear‑algebra/image filters sometimes improve (nogil > CPython 3.9 but not 3.11), decision tree and some ML runs are slower. TensorFlow/PyTorch aren’t tested because compatible builds aren’t available.
AI-written summary · about
chapters (20)
- 0:00Introduction
- 1:09What is the GIL
- 4:03Why Python’s GIL exists
- 6:09Introducing No-GIL Python
- 7:36Why No-GIL speeds Python
- 8:39Challenges in No-GIL design
- 11:51Thread safety in No-GIL
- 13:38CPython API changes
- 14:40New memory allocator
- 15:41Garbage collection in No-GIL
- 16:04Testing No-GIL on science
- 17:26Experiment design
- 19:13Results for Support Vector Machine
- 20:58Clustering results
- 21:42Decision tree results
- 22:03Linear algebra results
- 23:49Neural network results
- 24:52Why No-GIL didn’t help
- 26:21Compatibility challenges
- 27:05Conclusion
related talks
29mTrying No GIL on Scientific Programming
29mPython's Infamous GIL
31mGetting Around the GIL: Parallelizing Python for Better Performance
43mGIL: What's the hassle and... should I care?
38mDemystifying the GIL
29mGoodbye GIL: Exploring the Free-threaded mode in Python
28mThe GIL is dead: PyPy-STM
30mFree threaded Python past, present and future