Delta Lake has competition

on YouTube ↗25m

" summary

Compares Delta Lake, Apache Hudi and Apache Iceberg using runnable Spark examples on small datasets to show what works in practice: Delta Lake provides a DSL with a single MERGE that atomically upserts and deletes; Hudi requires a separate observe-then-delete flow and mixes metadata columns with user data; Iceberg supports time travel to previous snapshots but needs explicit table setup and Hive metastore wiring. The talk concludes Delta is the most mature, Hudi is feature-rich, and Iceberg is the hardest to integrate; example code is available on GitHub and the demonstrated operations produced the expected outputs.

AI-written summary · about

" chapters (7)

1. 0:00Start
2. 2:51The purpose of these tools
3. 3:18Their common points
4. 4:21In this presentation
5. 5:24Use case: Upsert and Delete
6. 10:49Apache Hudi
7. 15:07Apache Iceberg

" related talks

• 22mThe Who, What, and Why of Data Lake Table FormatsAndrew Madson · Commit Your Code 2024
• 45mBuilding an Open Data Lakehouse using Apache Spark, Apache Iceberg and DremioMike Flower · SQLBits 2023
• 25mBuild open format data lakehouseSatish Mane, Rajeev Jahswal · Conf42 Cloud Native 2023
• 31mBuilding large scale, transactional data lakes with Apache HudiNishith Agarwal · Big Data & AI Toronto 2020
• 34mApache Iceberg.Dipankar Mazumdar · Scale By the Bay 2023
• 01h 12mHands-on with Apache IcebergAnders Bogsnes · PyData London 2025
• 45mBuilding an Open Data Lakehouse using Apache Spark, Apache Iceberg and DremioMike Flower · SQLBits 2019
• 31mSimplifying upserts and deletes on Delta Lake tablesPrashanth Babu · Berlin Buzzwords 2021