Unlearning SQL
summary
Claim: treat SQL as a design language and implement Select/From/Where/GroupBy/Having pipelines in Python to avoid database overheads for transient data. Implementation details: use itertools.product or generator-based filters and maps, implement joins as lookup join (hash map lookup) or sort‑merge via external sort, do grouping into a defaultdict of lists then apply aggregates, and benchmark against SQLite3 or DuckDB — in many cases the in‑memory Python pipeline runs simpler and faster.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 1:05SQL's overuse
- 3:35Transient data in databases
- 4:58SQL clauses in Python
- 7:48Query optimizations in Python
- 9:14SQL joins in Python
- 13:07Sort merge join in Python
- 14:54Lookup join in Python
- 16:00Group by and having in Python
- 19:16SQL as design language
- 23:21Stop using SQL for transformation
- 25:11Q&A
related talks
23mOptimizing SQL + Python Pipelines for Data Science
32mBecause SQL is everywhere but you don't want to use it
27mBecause SQL Is Everywhere
01h 28mSQLAlchemy and You - Making SQL the Best Thing Since Sliced Bread
39mPython and SQL: Better Together, Powered by DuckDB
37mAccelerating Python Analytics by In-Database Processing
35mQuerying the queries: SQL Metaprogramming in Python
44mNonrelational SQL