Big data as we know it @bol.com
summary
Bol.com moves its BI from an Oracle monolith to a hybrid stack and reports concrete work: a 26-node Hadoop cluster used alongside Oracle and HBase, an in-house message bridge called eddie framework that turns service XML into HBase inserts, monitoring tools (Sniff) that aggregate per-flow MAPREDUCE stats, a Kyoto comparator for cross-source diffs, and an h2o loader that auto-generates Pig/Sqoop ETL for services; prototypes run in production and reduce per-service ETL effort.
AI-written summary · about
chapters (14)
- 0:00Introduction to bol.com
- 1:07Company growth and challenges
- 3:55Scaling with microservices
- 7:24Traditional data warehouse limits
- 10:12Hadoop infrastructure at bol.com
- 12:01Technical architecture overview
- 16:12Data design patterns in Hadoop
- 18:38Monitoring and data quality tools
- 22:29H2O loader framework
- 25:19Bulk interface access tools
- 26:46Bulk stats for data insights
- 28:32Lessons learned from Hadoop
- 29:55Future of bulk interfaces
- 31:39Q&A session
related talks
32mBusiness Intelligence in Microservice architecture #bbuzz
53mBig Data: from mammoth to elephant
47mSolving Real & Big (Data) Problems Using Hadoop
02h 05mBig Data and the Data Lake from the Analyst Perspective
57mBig Data: Threat or Opportunity?
49mData Platform Architecture, Evolution, and Philosophy
15mThe Future of Data
45mOur way to a data-driven enterprise - A Journey by Bosch