Data wrangling like a boss with Duckdb
summary
Claims that DuckDB reads a messy CSV of 34 million rows and six columns in seconds, summarizes it, and converts the 800 MB CSV to a 75 MB parquet in one streaming step. Shows the world’s most advanced CSV reader, parallel vectorized query engine, and range requests to read parquet on S3 without full download. Demonstrates a single-file transactional format that supports atomic updates and many tables and runs queries under a second.
AI-written summary · about
chapters (11)
- 0:00DuckDB intro and data wrangling
- 3:18What is DuckDB
- 9:13DuckDB philosophy
- 9:36CSV frustrations solved
- 16:23CSV to Parquet
- 20:59Complex queries on big data
- 21:59DuckDB file format
- 24:25Single-file storage
- 26:54Views and sharing files
- 28:46Language integrations
- 30:11Conclusion
related talks
58mData Wrangling [for Python or R] Like a Boss With DuckDB
37mDuckDB: Crunching Data Anywhere, From Laptops to Servers
40mDuckDB: Harnessing in-process analytics for data science and beyond
01h 02mAnalytics for not-so-big data with DuckDB
34mData Wrangling with DuckDB
27mThe Duck and the DataFrame: A Data Engineer’s Journey with DuckDB
33mData preparation and analytics on your laptop with DuckDB
28mAnalyze data easily with duckdb - and the implications on data architectures