Keynote Day 2 - Full
summary
Announces include Uniform GA with 4+ exabytes loaded and Delta processing cited at ~9 exabytes, plus a 73% BI performance uplift over two years; DuckDB adds native Delta via the Delta Kernel and releases 1.0; small language models are trained by iterative overgeneration and filtering to produce data that can beat GPT‑3.5 on summarization; an n‑gram suffix‑array index serves trillions of tokens at ~5.5 ms lookups and 60M API calls so far; Unity Catalog is open sourced, Lakehouse Federation and Lakehouse monitoring go GA, Delta Sharing has 16k recipients (40% off‑platform), Marketplace hits 2k listings, and Lakeflow: 3.5× faster, 30% cheaper consolidates ingestion, incremental SQL transforms and orchestration with real‑time triggers and materialized‑view pipelines.
AI-written summary · about
chapters (13)
- 0:00Opening remarks
- 3:13Professor Yin and small language models
- 21:59Day 1 recap and Day 2 transition
- 22:20Apache Iceberg and Delta Lake partnership
- 33:41DuckDB and Delta Lake integration
- 39:22Delta Lake 4.0 features
- 44:17Unity Catalog open source
- 1:05:12Unity Catalog metrics and BI
- 1:24:07Delta Sharing collaboration
- 1:36:20Texas Rangers data win
- 1:57:09Apache Spark 4.0 Python improvements
- 2:01:21Posit and sparklyr for R
- 2:14:06LakeFlow Connect pipelines
related talks
20mKeynote Review
14mKeynote Analysis Day 2
17mKeynote Analysis
23mKeynote Analysis
14mTony Baer, dbinsight & Doug Henschen, Constellation Research | Databricks Data + AI Summit 2023
01h 05mKeynote: Builders Day
01h 25mDay 2 Keynote: Bringing the Cloud to You
32mDesigning Data Infrastructure in the Age of Generative AI