Air Traffic Control (Data Governance) with Databricks Unity Catalog, tips, tricks and best practices
summary
Unity Catalog centralizes access for tables, files, models and external catalogs, adds column-level masking and row-level filters rolling GA this quarter, and enforces access for BI via AAD passthrough; audit and system Delta tables log usage for a provided Power BI pbit. Behind the scenes a predictive optimization service runs nonblocking optimize/vacuum on serverless compute, lineage is captured automatically from Spark jobs, and streaming tables/materialized views plus Delta sharing and lakehouse federation are available in preview.
AI-written summary · about
chapters (10)
- 0:00Introduction to Unity Catalog
- 1:04Unity catalog capabilities
- 7:10Access control demo
- 13:54Compute and system tables
- 21:46Advanced data sharing
- 25:14Performance and optimization
- 33:55Lineage and data quality
- 39:25Enterprise governance strategy
- 44:56Migration paths
- 48:48Q&A and closing
related talks
30mFederated Data, Centralized Action: A Governance Model Powered by Lakewatch and Databricks System Tables
05mUnity Catalog Demo of New Features
42mDriving Data Democratization
25mLakehouse + Graph: Delta Lake, Unity Catalog and Neo4j for Governed AI Pipelines
36mDatabricks as the Foundation for GenAI: What’s New and What Works
55mDemocratize data: The intersection of governance and analytics
49mAnalytics in SQL Server and Azure SQL Database
30mFrom Data Lake Entanglement to Data Mesh Decoupling: Scaling a Self-Service Data Platform