Polars GPU acceleration
summary
Polars reaches 1.0 and now wires GPU acceleration with cuDF into the optimizer by inserting a GPU callback after the Polars IR, producing a hybrid CPU/GPU fallback engine that preserves identical semantics. Benchmarks from NVIDIA show variable benefits — IO‑bound queries barely change while compute‑bound ones get up to 13× speedup. The integration uses cuDF to materialize subplans and returns Arrow-like data to Polars, and the approach works though gains depend on query shape and IO limits.
AI-written summary · about
chapters (10)
- 0:00Polars and GPU intro
- 1:50Lazy evaluation in Polars
- 5:23GPU design and integration
- 10:28Logical to physical execution
- 13:38GPU benchmarks and results
- 15:22Polars future with engines
- 17:51In-memory fast processing
- 19:36Streaming for large datasets
- 24:57Polars Cloud and distributed
- 27:49Q&A on plugins and streaming
related talks
49mKeynote on Polars Plugins
30mIs the great dataframe showdown finally over? Enter: polars
58mFrom Pandas to Polars Upgrading Your Data Workflow
38mExpressive and fast dataframes in Python with polars
28mAdvanced Polars: Lazy Queries and Streaming Mode
28mExpressive & fast dataframes in Python with polars
36mHow Narwhals brings Polars, DuckDB, PyArrow, & pandas together
29mcudf.pandas: The Zero Code Change GPU Accelerator for Pandas