Delta Lake and Delta Sharing
summary
Delta Lake provides a transactional layer on top of Parquet files stored on S3/HDFS, adding ACID transactions, DML (DELETE, UPDATE, MERGE), time travel, and CDC via table properties; it uses JSON commit logs with periodic Parquet checkpoints to scale metadata reads and supports concurrent writers and streaming reads. Delta Sharing is an open‑source server that shares data without copying by issuing short‑lived signed URLs and serving subsets of Parquet files for parallel download, with client support for Python/pandas and Tableau.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 1:03Traditional warehouse problems
- 3:54Delta Lake overview
- 8:29Delta Sharing intro
- 13:22Delta Lake demo
- 26:58Delta Sharing demo
- 29:22Conclusion & Q&A
related talks
39mDelta Lake: Diving into Data Lakes Without the Downsides
15mDelta sharing: An open protocol for secure data sharing
58mBuild reliable data lakes with Delta Lake & Databricks
38mGetting Started with Delta Lake
31mDelta Lake: Open Source Reliability and Quality for Data Lakes
18mDelta Lake – The Foundation Of Your Lakehouse
20mDelta Lake: Turn your Data Lake into a Lakehouse
33mThe Pace of Innovation in Delta Lake