Live Q&A Session #2
summary
A live Q&A covers concrete wins and limits: a corporate data graph faces idiosyncratic systems, needs precise accuracy and coverage metrics, and solves boundary expansion by merging partial events with human adjudication in loops that run days to weeks. DuckDB is an in‑process engine suited to interactive workloads up to a few terabytes while MotherDuck adds collaboration, security and Iceberg/data lake support. Beam ML runs batch and stream, shows up to 3x GPU efficiency and fleet gains from ~20% to ~60%, and exposes query and object APIs plus subfield and row predicate access control.
AI-written summary · about
chapters (16)
- 0:00Final Q&A
- 0:24Biggest challenges building data graph
- 2:30DuckDB as data warehouse
- 3:35GPU efficiency in Beam ML
- 4:39Policy decision point interface
- 5:44Iceberg Delta Lake in DuckDB
- 7:07Usability overhead in Beam ML
- 8:32Iterative discovery in data graph
- 10:44Security of edge data storage
- 11:46Beam ML for data at rest
- 12:49Row and column access control
- 14:14DuckDB use cases
- 15:57Beam with Spark
- 17:46Data flow and identity events
- 19:32Beam ML beyond Python
- 20:55Closing remarks
related talks
51mMotherduck: Data Analytics in the Post Big-Data Era
38mFrom Big Data to DuckDB: Ryan Boyd (MotherDuck) on Hyper-Tenancy, Duck Lake, and Making Data Fun.
28mAnalyze data easily with duckdb - and the implications on data architectures
24mData infrastructure to build bigger with less
26mBuilding Data Lake Using DuckDB
55mDuckLake: Learning from Cloud Data Warehouses to Build a Robust “Lakehouse”
06mA quack at building scalable data pipelines with DuckDB
07mAerodynamic data models: Flying fast at scale with DuckDB