Python in Bazel
summary
NVIDIA teams report concrete fixes for Bazel + Python: they add a pip-based lockfile that generates per-wheel BUILD repos, host a prebuilt wheel cache to avoid minutes-long rebuilds, and implement a py2/py3 dependency proxy so one requirements file selects appropriate wheels. Wheels prove non-deterministic; hosting prebuilt artifacts works but isn’t hermetic. Name collisions force either extracted fake repos or an import override hack that tries multiple packages at import time. Migration of ~100 data scientists succeeds.
AI-written summary · about
chapters (11)
- 0:00Python in Bazel challenges
- 1:10Python and Bazel integration
- 3:58Python package dependencies
- 7:06Improving Python UX with Bazel
- 10:12Python 2 and 3 compatibility
- 13:20Python wheels deterministic
- 15:51Python package name collisions
- 20:05Hidden build dependencies
- 22:33Entry points and non-Python code
- 24:16Future plans and upstream
- 26:03Summary and call to action
related talks
07mLightning Talk – Python Dependencies the Right Way
18mReproducible Builds with Bazel
09mBuilding a Hermetic Python Environment with Conda in Bazel
12mChallenge Accepted: Python Packaging Infrastructure for the RISCV64 Ecosystem
40mBuilding a binary extension
26mPhantom Dependencies: is your requirements.txt haunted?
09mLightning Talk – A short overview of our company's journey to sandbox Python
09mPython and Bazel: Aspect’s rules_py