Building an Open Data Lakehouse using Apache Spark, Apache Iceberg and Dremio
summary
Shows an Open Data Lakehouse built with Apache Spark to create Apache Iceberg tables and Dremio to query and manipulate them in place; demonstrates create/insert/merge SQL working against Iceberg, Iceberg time travel via snapshots, and exposing the same data to Tableau through Dremio with JDBC/ODBC. Builds a small Iceberg table in Spark, adds rows, then updates and merges from Dremio — the demo works end-to-end — and highlights virtual data sets, lineage, security options and a free Community Edition.
AI-written summary · about
chapters (10)
- 0:00Start
- 1:26What is an "Open Data Lakehouse"?
- 2:49The evolution to the Data Lakehouse
- 9:24Table format adoption
- 11:20How Is Dremio Different From Traditional SQL Engines?
- 13:48Typical components of an Open Data Lakehouse
- 16:56Using Dremio with Iceberg - CREATE TABLE
- 18:03Using Dremio with Iceberg - MERGE new data
- 19:55Demo
- 33:53From Data Warehouse to Data Lakehouse
related talks
45mBuilding an Open Data Lakehouse using Apache Spark, Apache Iceberg and Dremio
13mBuild your open data lakehouse with Dremio and AWS (PRT085)
01h 12mHands-on with Apache Iceberg
50mDesigning a scalable data platform based on Apache Spark/Iceberg
39mFlexible OpenSearch Data Management With Apache Iceberg: Data...
25mBuild open format data lakehouse
46mTake the Polar Plunge: A Fearless Introduction to Apache Iceberg™
49mHow Apache Iceberg Really Works