Apache Iceberg: An Architectural Look Under the Covers
summary
Explains that Apache Iceberg stores per‑snapshot metadata files that list exact data files via manifests and manifest lists enabling snapshot isolation and time travel, uses manifest and column stats to prune millions of parquet files so planning falls from 9.6 minutes to ~10–25 seconds and queries from hours to ~42 seconds, supports schema evolution and small CRUDs, provides Java Python Go Rust C++ libraries for interoperability with engines like Trino DuckDB Dremio and is a spec not a storage engine minmax and partition filtering
AI-written summary · about
chapters (10)
- 0:00Apache Iceberg intro
- 2:11Table formats and Iceberg need
- 9:08Hive format limits
- 13:40How Iceberg solves it
- 16:27Iceberg table architecture
- 21:21Iceberg transactions demo
- 26:39Iceberg performance at Netflix
- 29:08Iceberg benefits and features
- 31:18Resources and next steps
- 32:41Q&A
related talks
52mApache Iceberg - An Architectural Look Under the Covers
31mHow Apache Iceberg Enables Multi-Engine Data Platforms
34mApache Iceberg.
41mTip of the Iceberg
49mHow Apache Iceberg Really Works
01h 04mAn Extremely Technical Overview of How Apache Iceberg Planning Actually Works
46mTake the Polar Plunge: A Fearless Introduction to Apache Iceberg™
01h 12mHands-on with Apache Iceberg