Bigtop 101
summary
Bigtop 1.3.0 builds Apache Hadoop ecosystem packages into RPMs and DEBs inside a containerized, reproducible build image using Docker (plus docker-compose and Ruby for multi-container tests), produces artifacts in a bigtop-output folder after long builds (often ~10 hours), and supports aarch64 out of the box. The workflow shows how to run a 5-container smoke test with a small YAML, and how to deploy via Puppet to multiple physical nodes (OpenJDK 8, Puppet modules, hostnames, disabled firewalls); the example completes and services report the expected HDFS/YARN nodes.
AI-written summary · about
chapters (8)
- 0:00Introduction to Bigtop
- 1:31What is Bigtop
- 2:57Bigtop components overview
- 3:58Setting up build environment
- 7:01Building Bigtop from source
- 9:34Running smoke tests
- 13:30Deploying on multiple nodes
- 21:45Q&A and closing
related talks
44mDeveloping for Big Data at the Intersection of Containerization and Infrastructure as Code
35mGroovy, There's a Docker in My Application Pipeline
52mContinuous Delivery with Containers: The Good, the Bad, and the Ugly
46mBuilding for Enterprise by Testing OSS: One Codebase to Rule Them All
51mJava applications containerized and deployed, make your buildpack edition
40mAutomated Path to Production using Open Source and VMware Tanzu
41mContinuous Delivery with Docker Containers and Java EE
14mBuild It Your Way: Modular Pipelines for Bootable Containers