Upgrading from the modern data stack to the modern data lake
summary
Argues that replacing a cloud data warehouse with an open data lake plus a federated query engine is modernization, not migration: use Iceberg/Delta/Hudi with ORC/Parquet/Avro, separate storage into land / structure / consume zones, run Trino-based queries with 60 TB fault‑tolerant queries, add a semantic/governance layer (Gravity) and automation (automatic data classification, data lake optimization), and adopt streaming ingest into Iceberg that lets tables be queried in under a minute while cutting S3 GET costs by up to 70%.
AI-written summary · about
chapters (7)
- 0:00The Modern Data Stack
- 1:43Redefining Modernization
- 4:33Building the Modern Data Lake
- 8:06Data Lake Architecture
- 10:58Introducing Starburst Galaxy
- 13:49Platform Layers and Features
- 17:34New Capabilities and Summary
related talks
51mHow to Build a Modern Data Lakehouse?
31mData Warehouses Meet Data Lakes
33mThe advent of the open data lake
41mModern Data Architectures – What They Are and How to Implement?
24mModernizing the Data Warehouse
11mBreaking Data Silos: Enterprise Data Management
34mData Lakehouse: Who, Why and How
41mOn Data: Sky above, sand below, peace within