This Impala not only reads, but modifies and optimizes Iceberg tables
summary
Impala adds writable Iceberg support: it implements row-level positional deletes (and can read equality deletes), rewrites DELETE/UPDATE as inserts into an Iceberg delete table, and routes rows to multiple sinks with a new merge sync so updates, deletes and inserts can be produced in one plan. Read performance avoids broadcasting by using directed distribution joins. Table maintenance adds OPTIMIZE compaction to merge small files and remove delete files, DROP PARTITION with predicates and transforms, plus ROLLBACK and EXPIRE SNAPSHOTS to reclaim space.
AI-written summary · about
chapters (8)
- 0:00Introduction and agenda
- 1:13Legacy Hive tables limitations
- 5:43Apache Iceberg overview
- 8:53Apache Impala architecture
- 12:33Modification techniques in Iceberg
- 17:10Delete update and merge statements
- 20:21Table maintenance operations
- 25:03Conclusion and Q&A
related talks
05mMaking Iceberg walk and talk like Postgres
13mExploring the Depths of Apache Iceberg's Metadata Capabilities
39mIntroducing Apache Iceberg: Tables Designed for Object Stores
41mTip of the Iceberg
01h 12mHands-on with Apache Iceberg
34mApache Iceberg.
31mHow Apache Iceberg Enables Multi-Engine Data Platforms
01h 04mAn Extremely Technical Overview of How Apache Iceberg Planning Actually Works