Future Data Systems Seminar Series (Fall 2025)
2025-09-22..2025-12-08
-- online --
who it's for
data engineers, lakehouse architects, storage engine developers, analytics platform engineers, database architects, streaming systems engineers, open source contributors
topics
Apache Iceberg planning; delete file indexes; real-time lakehouse ingestion; columnar transfer with Apache Arrow ADBC; Vortex columnar file format; Apache Hudi transactional storage; DuckLake SQL metadata pruning; multi-statement lakehouse transactions; deletion vectors and index-LSM; Polaris server-side Iceberg catalog; Apache Fluss streaming table store; Iceberg caching for low latency
| title | author | length |
|---|---|---|
| An Extremely Technical Overview of How Apache Iceberg Planning Actually Works | Russell Alexander Spitzer | 01h 04m |
Detailed walkthrough of the Java implementation of Apache Iceberg planning shows how Spark SQL push predicates convert to iceberg expressions, how… | ||
| Apache Hudi: A Database Layer Over Cloud Storage for Fast Mutations & Queries | Vinoth Chandar | 01h 08m |
Apache Hoodie implements a storage engine over cloud object storage with file groups with base + log files and a timeline stored as an LSM to support… | ||
| DuckLake: Learning from Cloud Data Warehouses to Build a Robust “Lakehouse” | Jordan Tigani | 55m |
DuckLake stores metadata in a SQL catalog so file pruning runs as SQL queries against ducklake_file and ducklake_column_stats instead of traversing JSON… | ||
| Vortex: LLVM for File Formats | Will Manning | 01h 02m |
Vortex is an open source, Rust-based columnar file format built on Arrow with plug‑in encodings (FastLanes, ALP, FSST, PCODC, optional Zstd), extensible… | ||
| Where We’re Going, We Don’t Need Rows: Columnar Data Connectivity with Apache Arrow ADBC | Ian Cook | 01h 04m |
ADBC is a multi‑language client API and driver standard that delivers query results in Apache Arrow columnar format instead of row rows, eliminating the… | ||
| SingleStore: Storage Metadata for Modern Cloud Databases | Joyo Victor | 59m |
SingleStore adds a centralized metadata DB — the bottle service — that records blobs, snapshots, log chunks, compute sessions and per-blob references in a… | ||
| Multi-statement Transactions in the Databricks Lakehouse | Ryan Johnson | 56m |
Databricks adds multi-statement, multi-table transactions to the lakehouse by moving commits through catalog-managed commits, staging unpublished commits and… | ||
| Mooncake: Real-Time Apache Iceberg Without Compromise | Cheng Chen | 01h 03m |
Mooncake adds a real‑time layer on top of Apache Iceberg to get fresh Postgres changes into analytics with subsecond ingestion by buffering writes in Arrow and… | ||
| Firebolt: Why Powering User Facing Applications on Iceberg is Hard | Benjamin Wagner | 01h 08m |
Firebolt adds Iceberg support but finds Iceberg’s hierarchical metadata and object storage latency create a ~0.5s baseline for cold queries, so Firebolt caches… | ||
| Reconstructing History with XTDB | Jeremy Taylor, James Henderson | 01h 06m |
XTDB v2 is a temporal HTAP database released this year that exposes SQL 2011 valid/system time semantics, stores immutable transaction events in object storage… | ||
| From Storage Formats to Open Governance: The Evolution to Apache Polaris | Prashant Singh | 01h 12m |
Snowflake engineers build and contribute Apache Polaris, an Iceberg REST catalog that shifts concurrency, conflict resolution and snapshot creation from… | ||
| Apache Fluss: A Streaming Storage for Real-Time Lakehouse | Jark Wu | 01h 01m |
Apache Fluss is a distributed streaming table store that promises millisecond reads and writes, mutable primary-key tables that emit append-only change logs… | ||
| nothing matches | ||