Data Warehousing Migrations: Lessons from Home Depot
summary
Home Depot describes a multi-year migration from an on‑prem EDW into Google BigQuery using GCS, Pub/Sub, Dataflow, App Engine and BigQuery as the core, staging via on‑prem Hadoop, streaming moved into Pub/Sub, and BI served by Tableau and AtScale. They adopt hierarchical slot reservations, keep natural keys, use nested structures and create domain “building blocks” plus materialized Analytical Data Sets, cut multi‑hour jobs to minutes, load ~800 TB of clickstream, and emphasize reskilling, an analytics enablement team, and copying schemas first.
AI-written summary · about
chapters (7)
- 0:00Introduction and keynote
- 7:05Home Depot’s analytics challenge
- 13:26Migration goals and architecture
- 27:59Technical implementation details
- 37:12Performance wins and results
- 41:19Lessons learned and change management
- 46:02Conclusion and future outlook
related talks
36mMigrating Our Data Warehouse to the Cloud
15mData strategy: Getting ready for Cloud migrations
30mData Warehouse in Google Cloud
43mData warehousing reinvented for today's needs
38mLesson learned implementing a large-scale data mesh transformation
27mModernize Your Data Warehouse
58mTales of Data Architecture Evolution
26mPutting your data warehouse to work - Reverse ETL & Operational Analytics