Building a Distributed Build System at Google Scale
summary
Google builds a distributed system called Build Rabbit that delegates compilation to Bazel and moves from a push model to a pull model using a persistent queue. It handles ~30,000 engineers, ~15,000 human and ~30,000 robot commits daily, ~5 million builds/tests and petabytes of artifacts. The key claims: resilience logic centralizes (thins clients), migrations occur with zero downtime through backend-first launches and staged rollouts, and the new architecture succeeds in preserving progress and improving robustness.
AI-written summary · about
chapters (25)
- 0:00Introduction
- 0:56What does build system do
- 2:28What it means to build and test
- 6:06Benefits of monolithic repositories
- 11:25Distributed build system
- 14:14Build Rabbit
- 16:43What is Distributed Build System
- 23:20Migration Considerations
- 24:40Mixed Mode
- 26:03Launch Friendly Clients
- 27:28Visibility
- 30:17Control Plane
- 31:06Old Architecture
- 33:21Questions
- 33:58monolithic repository
- 34:43Supported languages
- 37:29Are they identical
- 38:01Source code repository
- 38:39Stateless build system
- 38:52Repeatable builds
- 39:50Internal dependencies
- 40:34Does the build system build OS and Android apps
- 41:17Does the build system define the artifact
- 42:37How do I make sure that I have the latest one
- 43:20Outro
related talks
39mDevelopment, Review and Integration at Google scale
23mPolyglot Builds with Bazel
11mPerformant Bazel Builds for Web Monorepos at Scale
34mBuilding at Google: From Idea to Production
31mBazel: Do We Really Need Another Build Tool?
06mLightning Talk: Do You Love or Hate Your C++ Build System?
59mDay 2 Keynote
41mDude, Where's My Build? Android CI/CD at Uber