The Evolution of Python Packaging
summary
Facebook builds a self-contained Python packaging system centered on par (Python archive): an executable shebang + zip that evolves into zipimport-backed fast-par, hard‑linked files in a content-addressable store (CAS), and a shared sparse/ESPAR layer to avoid duplicate libs. Key tricks: open-as-FD via a readlinke hack to survive replacements, pre-hash to cut launch time ~25%, and a prototype czar (squashfs+FUSE) to eliminate zipimport. System runs at hundreds of thousands of launches and is incrementally successful in reducing disk, network and startup costs.
AI-written summary · about
chapters (14)
- 0:00Intro
- 2:06Why Package
- 3:45Modest Proposal
- 5:51What is Par
- 6:19Par Version 1
- 10:38Everythings Broken
- 11:20Failures
- 12:37TempWatch
- 14:51CAS
- 17:15Sparse
- 20:22Versions
- 21:34Polytree
- 24:21Python 25 Prehash
- 25:06Python 26 Czar
related talks
24mInside the Cheeseshop:How Python Packaging Works
30mThe Evolutionary Saga of Python Packaging
41mPrecompiling and deploying Python to any Linux, Mac OS X or FreeBSD system
44mNobody Expects the Python Packaging Authority
02h 36mPython Packaging Demystified
36mPython Packaging from Init to Deploy
25mInside the Cheeseshop: How Python Packaging Works
32mInside the Cheeseshop: How Python Packaging Works