Tensorflow with Bazel — Lessons Learned
summary
TensorFlow uses Bazel at scale and documents concrete fixes: dependency downloads from 20+ external repos cause flaky builds so the team adds a Bazel downloader with mirrors and retries to reach >99% reliability; CUDA support uses fake repos with symlinks to user-installed system packages; custom Skylark rules build Swig and protoc locally; a configure script writes bazelrc files to hide dozens of knobs; dynamic shared libraries require workarounds; distributed builds cut local build time from ~40–50 minutes to ~5–10 minutes.
AI-written summary · about
chapters (14)
- 0:00TensorFlow and Bazel intro
- 1:05Pre-open-source Bazel at Google
- 3:54Challenges after open-sourcing
- 5:20Managing Bazel dependencies
- 7:30Reliable dependency downloader
- 8:14Missing build files handling
- 9:20CUDA system packages limits
- 11:06Building Swig and Protobuf
- 13:17Multi-platform build configs
- 15:50Dynamic linking challenges
- 18:47Packaging TensorFlow
- 20:33Open Bazel adoption problems
- 21:41Distributed builds impact
- 22:45Summary and lessons
related talks
38mBuilding Self Driving Cars with Bazel + Q&A
25mThe status of Tensorflow on Arm-based HPC
43mTensorflow on open source GPUs
02mInterview
37mSimplifying Training Deep & Serving Learning Models...using Tensorflow
43mDeep learning beyond the learning
36mHigh-Performance Distributed Tensorflow
31mSupercharging Android Apps with Tensorflow