Covering indexes in Adobe Experience Platform Data Lake with Hyperspace
summary
Adobe Experience Platform runs Iceberg tables and adds Hyperspace indexes to cut query latency—benchmarks show up to ~20–25x speedups for Adobe workloads (Microsoft reports ~2x average, up to 10x). The demo builds a covering index (id as index column, name included), stores index files where configured, buckets by key (200 buckets default), and uses a hybrid scan that reads index files plus recently appended data. APIs shown: create/refresh (full/incremental/quick)/optimize/vacuum; current gaps: nested fields and ORC/Avro/Hoodie support.
AI-written summary · about
chapters (5)
- 0:00Start
- 1:25Overview of Adobe Experience Platform Data Lake
- 8:03Benchmarks
- 16:00Consistency with Hybrid Scans
- 35:15Gaps and Roadmap
related talks
27mBuilding a PostgreSQL data warehouse
34mApache Iceberg.
32mIntroducing Oracle Autonomous AI Lakehouse Open Multicloud Data Platform
13mBuild your open data lakehouse with Dremio and AWS (PRT085)
27mHive-Iceberg - Breaking the Ice: A Closer Look at Hive-Iceberg
45mBuilding an Open Data Lakehouse using Apache Spark, Apache Iceberg and Dremio
52mApache Iceberg - An Architectural Look Under the Covers
31mHow Apache Iceberg Enables Multi-Engine Data Platforms