Building Apache Heron
summary
Describes integrating Apache Heron into a Bazel build that produces a self-extracting install script for CentOS, Ubuntu and Darwin, resolves a Storm/Eco dependency cycle using neverlink, and packages Java/C++/Python artifacts through layered package_tars and a launcher genrule. Claims support for topologies handling up to 60,000 tx/s with low-latency SLAs, reports cutting releases from ~45–60 minutes toward remote caching speeds (~5 minutes), and notes Bazel version upgrades sometimes break the build.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 1:09Apache Heron origins
- 3:37Heron topology terms
- 4:59Heron topology structure
- 6:25Basil multi-language support
- 8:13Eco topology design
- 9:15Eco integration challenges
- 11:24Basil build scripts
- 13:38Basil build process
- 15:22Heron install script
- 17:11Dependency graph visualization
- 18:13Basil and Heron integration
related talks
34mBuilding Real-time Systems with Bazel
52mProcessing Billions of Events in Real-Time with Heron
11mRemote execution with 100 to 100,000 CPUs
50mProcessing Billions of Events in Real-Time with Heron
39mTwitter Heron in Practice
29mUnlocking Bazel's Potential with Monorepo: A Key to Efficient Software Development
30mBuilding a Distributed Build System at Google Scale
20mLow latency scalable web crawling on Apache Storm #bbuzz