How Bazel Handles Globs
summary
Bazel exposes file existence to build code via glob patterns but historically evaluates globs twice, causing long stalls and high memory from per-directory graph nodes; a 2017 dynamic analysis approach was reverted, and a flag exists to prefetch/warm large filesystems. The current implementation rewrites recursive glob traversal as a single state machine that collapses local Skyframe nodes, saving memory, while Skyframe still provides incremental correctness; future paths include Java virtual threads to pause execution and avoid double evaluation.
AI-written summary · about
related talks
33mEmpyrean Evaluation
28mBazel Remote Execution API vs. Goma + Q&A
08mLightning Talk – Creating Java based Source Code Generators for Bazel
26mSkylark: The Bazel Extensibility Language
21mBonsai: Compiling Queries to Pruned Tree Traversals
16mOptimistic and Scalable Global Function Merging
24m(Ab)using Compiler Tools
55mNew Bytecodes, New Objects