Data in the Apache "big data" ecosystem
summary
Argues that most enterprises do not need massive clusters: a Forrester survey of ~1,800 firms shows less than 10% at petabyte scale and typical corporate datasets sit under ~80 TB, so focus is on integration not raw storage. Recommends using Apache Bigtop to package, test and deploy complex stacks (he built an ODP reference implementation with it in days), ties to orchestration like Juju and Docker, and validates 30+ components across ARM and OpenPower as well as x86.
AI-written summary · about
chapters (11)
- 0:00Introduction and joke
- 1:41MapR and in-memory databases
- 3:08Big data hype and reality
- 5:59Data size and storage trends
- 8:06Data challenges and Apache tools
- 10:16Apache ecosystem overview
- 12:42Choosing the right components
- 13:48Complexity in Apache projects
- 14:30Apache BigTop explained
- 17:44BigTop storage and deployment
- 18:47BigTop vs other tools
