Expressive and fast dataframes in Python with polars
summary
Polars uses a Rust core over Arrow and shows a lazy expression planner that fuses operations and parallelizes independent aggregates, letting a cloud demo scan a 1.8 GB CSV of Stack Overflow posts without fully loading it, group tags into a list[str] column, semi-join and filter for python tags, and surface top-voted python questions (median score 1.78). Recent streaming work even reads 80 GB on 16 GB RAM so out‑of‑core and multicore speedups actually work.
AI-written summary · about
chapters (15)
- 0:00Introduction to polars
- 2:11Pandas success and limits
- 6:07Alternatives to pandas
- 8:36Apache Arrow as memory format
- 10:01What is polars
- 13:11Demo setup and data load
- 17:51Exploring polars expressions
- 22:08Handling list columns
- 23:16Lazy evaluation for big data
- 27:32Top Python questions with polars
- 31:52Memory and streaming
- 33:18Polars future and distributed
- 36:15When to use lazy
- 37:20SQL and index notes
- 38:02Summary
related talks
28mExpressive & fast dataframes in Python with polars
30mIs the great dataframe showdown finally over? Enter: polars
58mFrom Pandas to Polars Upgrading Your Data Workflow
25mIntegrating Rust with Python...
01h 32mAn Introduction to Polars
40mBeyond Pandas: The Great Python Dataframe Showdown
01h 16mPower up your Polars code with Polars extention
28mAdvanced Polars: Lazy Queries and Streaming Mode