Build open format data lakehouse
summary
Shows how to build a cloud‑native lakehouse on S3 with Glue catalog plus Athena, Redshift Spectrum and EMR, and compares three open table formats: Apache Hudi, Apache Iceberg, Delta Lake. Hudi is pitched for streaming and CDC with built‑in indexing and auto file sizing, Iceberg for snapshot/time‑travel and partition evolution, Delta for deep Spark integration and transaction logs; notes practical ops like checkpointing, compaction/vacuum/optimize and which services support read/write, concluding these combos deliver ACID, time travel and governance.
AI-written summary · about
chapters (9)
- 0:00Data lakehouse on AWS
- 3:05Traditional analytics limitations
- 6:16Serverless data lake benefits
- 7:21Lakehouse challenges and solutions
- 9:32Open table formats
- 11:57Lakehouse options on AWS
- 15:11Hudi vs Delta Lake vs Iceberg
- 22:38Choosing table format
- 24:25Q&A
related talks
22mThe Who, What, and Why of Data Lake Table Formats
37mBuild a Modern Data Platform on AWS
41mTransactional data lakes for your analytical workloads
40mUnbundling of the Cloud Data Warehouse: Open Source Databases and Data Lakes
34mData Lakehouse: Who, Why and How
31mPanel: Open Data Lakes
34mCatalogs, Iceberg, and Delta: Building the Foundation for Lakehouse Analytics
47mConstruyendo Lagos de Datos y Analíticos en AWS + Datos Abiertos