Behind the Hype - Architecture Trends in Data
summary
Argues that modern lakehouses using Delta lakehouse (transaction log, time travel, updates/merges) actually fix old lake pains and enable warehousing on cheap files but trade maturity and some SQL features; warns of many ruined early lake projects. Says data mesh skepticism is merited: sound ideas about domain ownership and data-as-product but most orgs lack maturity and vendors oversell tech as a turnkey fix. Notes reverse ETL tools solve pushing warehouse data into apps, and DBT-driven analytics engineering codifies testable, repeatable modeling.
AI-written summary · about
chapters (10)
- 0:00Introduction and speaker background
- 1:51Understanding hype and Gartner curve
- 8:24Healthy skepticism checklist
- 12:32Data architecture cycles
- 17:03Lake houses explained
- 25:22Data mesh deep dive
- 32:47Engineering in data roles
- 39:04Buzzword bingo roundup
- 47:52AI and ChatGPT trends
- 49:14Closing thoughts and Q&A
related talks
54mBehind the Hype - Architecture Trends in Data
28mYour data lakes need architecture too!
31mData Warehouses Meet Data Lakes
41mModern Data Architectures – What They Are and How to Implement?
44mWhat's the Best Big Data Architecture for You?
52mData Warehouse, Data Lake, Data Mesh, Oh My: The History of Data Storage
28mLakehouse Architecture with Delta Lake and Databricks
24mData Mesh and Lakehouse