How open source is re-shaping the cloud data warehouse landscape
summary
Argues the cloud warehouse era is fragmenting: ClickHouse (open source) hits millisecond interactive queries vs seconds on cloud warehouses, and a reproduced benchmark shows Snowflake remains about 3–5x more expensive after heavy tuning. Describes building the ClickPy demo, measuring terabyte loads, using double‑write into an object store plus a realtime DB, federated CSV reads vs Parquet imports, the ClickBench public benchmarks, and materialized views into the lake for hybrid queries with Iceberg or Delta Lake.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 1:52History of data warehousing
- 6:48What are data lakes
- 10:28What are analytical databases
- 15:35Why Snowflake
- 17:53Why Data Lakes
- 22:47How to get started
- 29:18Summary
related talks
28mRethinking Status Quo: Open Source and the Cloud Data Warehouse
40mUnbundling of the Cloud Data Warehouse: Open Source Databases and Data Lakes
40mTo Warehouse or to Lakehouse? And How Do Various Open Source Project Compare?
17mBuilding Real-Time Data Lake with ClickHouse
28mThe architecture behind modern cloud data warehouses
23mThe Open Source Analytic Universe, Version 2022
43mBreaking Down Data Warehouses: What to Know Before You Choose
28mColumnstore Revolution