DataEngBytes 2023 - MEL-T1-07 - Ryan Boyd
summary
Argument for replacing monolithic cloud warehouses with a hybrid local+cloud model that actually runs: a 20‑MB in‑process SQL OLAP engine (DuckDB) embeds in Python, JS/WASM and the browser, reads parquet/CSV/HTTP and registers pandas DataFrames, and a cloud product (MotherDuck) runs the same engine client and server so the planner splits work. Benchmarks show parquet scans on a laptop at 2.95s, native format at 1.5s, and a cloud run hitting 1.45B rows in 145ms. browser WASM engine and hybrid client/server query planning enable local joins and cached, SQL-powered filtering.
AI-written summary · about
chapters (11)
- 0:00DuckDB introduction
- 2:51What is DuckDB
- 6:45Why DuckDB popular
- 10:36DuckDB analytical workloads
- 13:10DuckDB storage compression
- 16:06DuckDB datasets
- 19:00DuckDB community growth
- 20:45MotherDuck introduction
- 23:40MotherDuck hybrid execution
- 25:07DuckDB local to cloud
- 26:12Q&A
related talks
30mDataEngBytes 2023 - BRI-T1-08 - Ryan Boyd
24mData infrastructure to build bigger with less
33mDataEngBytes 2023 - SYD-T1-01 - Ryan Boyd
40mDuckDB: Harnessing in-process analytics for data science and beyond
28mAnalyze data easily with duckdb - and the implications on data architectures
27mIn Process Analytical Data Management with DuckDB
51mMotherduck: Data Analytics in the Post Big-Data Era
40mDuckDB beyond the notebook