How I Put a Rocket Engine into Python Porting ClickHouse for High Performance App
summary
A drop-in tool wraps a ClickHouse OLAP engine under the pandas API so you can pip install chdb and run with same pandas code, one-line change; it uses zero‑copy reads, lazy SQL pushdown across CSV/JSON/S3/Postgres sources, and streaming chunks to avoid OOM. Claims include up to 247x faster, 7.7× speedup on a 50M join+aggregate test, 20–24% faster 1M-row export, and up to 23× less peak memory; it runs in Lambda, workers, and multiple language bindings.
AI-written summary · about
chapters (10)
- 0:00CHDB and ClickHouse intro
- 3:00What is CHDB
- 7:02How CHDB works
- 10:16Language & deployment
- 11:42Benchmarks vs pandas
- 14:43Pandas compatibility
- 16:31NL to pandas with CHDB
- 19:24Python 3.14 threading
- 21:10Enterprise features
- 22:01Q&A: CHDB vs Polars
related talks
40mScaling Python - Bank Edition
01h 06mBodo: Automatic HPC Performance and Scaling for Data Processing in Python
40mPython and pandas as back end to real-time data driven applications
26mBodo: Supercomputing-Like Performance and Scale for Python/Pandas
30mPython and the AI Lakehouse
37mAccelerating Python Analytics by In-Database Processing
22mIt’s all about the db - a journey to accommodate 100k events per second
28mBridging Python and Lakehouse worlds with Arrow Flight and DuckDB.