Architecting Native Python Extension Modules
summary
Over 20% of the most downloaded Python modules use native extensions, and the core claims are concrete: crossing the FFI is expensive so batch operations or move hot loops into native code, the GIL becomes easier to avoid once Python objects are out of the hot path, and packaging requires building wheels for each OS/arch/Python. Recommended tools include pyo3 and pybind11, use a sampling profiler like ppy, separate Python-facing API from core logic into distinct packages or crates, and one example moved a Levenshtein hot loop into Rust and processed tens of thousands of files with large speedups.
AI-written summary · about
chapters (7)
- 0:00Introduction to native extensions
- 1:06What are native extension modules
- 2:31When to use native extensions
- 5:23Performance with FFI calls
- 6:46Architecture patterns for extensions
- 8:33Key takeaways and summary
- 8:53Q&A session
related talks
29mThe Art of Extending Python with other languages
53mRethinking How We're Linking: What to do when speeding...
31mPumping up Python modules using Rust
01h 02mWriting Python Bindings for C++ Libraries: Easy-to-use Performance
29mC++ ♥ Python: from modules to hybrid applications
33mBuilding with Multiple Languages: A Python-First Approach
36mPumping up Python modules using Rust
25mHow to bring your C++ project to Python land