Polars Plugins: how you (yes, you!) can extend Polars Dataframes
summary
Claim: writing a Polars plugin is practical and yields real speedups. Workflow uses a cookiecutter template, Polars plus maturin, Rust for the UDF, register-from-Python and compile; the demo builds a name-by-location pastry classifier using a crates.io geospatial library and polygon contains checks, it runs from Python with no Rust knowledge required for users. Live demo shows lazy execution cutting calls from 627 vs 40 loops, proving plugins integrate with optimization and the example actually works.
AI-written summary · about
chapters (8)
- 0:00Polars plugins superpowers
- 3:14Polars plugins game changers
- 5:23Six steps to build plugin
- 6:08Live coding plugin from scratch
- 15:22Plugins fit lazy execution
- 18:56Real-world plugin use cases
- 19:37Recap and takeaways
- 21:11Q&A on plugins
related talks
30mPandas 2, Dask or Polars? Tackling larger data on a single machine
29mHow Narwhals is silently bringing pandas, Polars, DuckDB, PyArrow, and more together
32mCYOP: Create Your Own Plugin
17mMaking Things Nice in Python
33mBuilding Data Science Tools for Sustainable Transformation
01h 55mAll Them Data Engines: Data Munging with Python circa 2023
34mpydiverse pipedag: A library for data pipeline orchestration
33mYou Don’t Need Spark for That: Pythonic Data Lakehouse Workflows