Evolving Data Governance With Unity Catalog
summary
Announces an open-source Unity Catalog (v0.1) now public on GitHub and accepted by the Linux Foundation, ships table, volume (unstructured) and AI tool/function support, and implements the Iceberg REST API for cross-format reads. Adds GA Lakehouse Federation (5,000 monthly active customers) to mount external sources, GA Lakehouse monitoring for incremental quality checks, and a preview of attribute-based access control. Claims customers run tens of petabytes (GM) and thousands of users (Pepsi) on it and demonstrates the repo live.
AI-written summary · about
chapters (10)
- 0:00Unity Catalog governance intro
- 1:04Modern data governance challenges
- 3:33Lakehouse Federation open connectivity
- 5:20Federation GA and Hive/Glue support
- 6:46Unified governance for data and AI
- 7:08Lakehouse monitoring and ABAC preview
- 8:55Why open source Unity Catalog
- 11:23Iceberg API compatibility
- 12:48Unity Catalog 0.1 release
- 14:12Future roadmap and collaboration
related talks
23mKeynote Analysis
26mJoel Minnick, Databricks | Databricks Data + AI Summit 2024
29mServing a multi-platform Data Lake via Delta, Iceberg, Unity and Glue
52mElevate your data and AI governance with Databricks Data Intelligence Platform
38mHerding Cat(alogue)s: The Apache Iceberg™ Catalog Landscape
49mUnity Catalogue and Purview: Data Governance Bedfellows
33mThe advent of the open data lake
23mAnalystANGLE