Gravitino: A multi-regional, geo-distributed meta datalake
summary
Gravitino presents a concrete open source metadata layer that just entered the ASF incubator today, connects to Hive, Iceberg, Postgres, Spark, Trino and Doris, and runs as single or multi-node geo-distributed servers that push subqueries to other nodes to avoid moving full datasets. The demo shows a web UI, REST, Java and Python APIs, and live cross-source SQL joins across catalogs; the project already has ~76 external contributors and hundreds of PRs and issues, and the demo mostly works despite a flaky local IP-based setup.
AI-written summary · about
chapters (9)
- 0:00Introduction to Gravitino
- 1:58Why Gravitino exists
- 3:02Gravitino as metadata catalog
- 8:01Recent releases and features
- 11:15Live demo of Gravitino UI
- 13:23Running cross-source SQL queries
- 16:36REST API and community updates
- 18:46Future of data governance
- 20:37Q&A and closing
related talks
34mCloud Agnostic Data Platform with DBT, Trino, Iceberg and MinIO
34mApache Iceberg.
33mDiscover How to Create Your Own Metadata-Driven ML Platform from Scratch
51mPresto Open Source SQL Engine for Massive Scale Data Lakehouses
31mHow Apache Iceberg Enables Multi-Engine Data Platforms
19mKickstarting a greenfield data project
14mMosaic: A Framework for Geospatial Analytics at Scale
29mLakeside AI: Bringing Models to Data with Open Lakehouses & Agents)