Hacking `import` for speed: how we wrote a GPU accelerator for pandas
summary
Project hacks Python’s import system to proxy import pandas to cuDF (RAPIDS), using a metapath importer and proxy DataFrame types that route supported operations to GPU and fall back to CPU. Claimed speedups range from 10–100× commonly and up to 150× in benchmarks, passes roughly 90–94% of pandas tests, and offers a Jupyter load_extension for zero-code switching. Limitations include GPU memory bounds, C‑API calls that often fall back, and some unsupported pandas corners.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 3:32What is Rapids CF
- 7:50CF pandas zero-code
- 13:06Hacking Python's import
- 23:17CF pandas limits
- 24:44Future of pandas
- 25:28Get started with CF
- 27:13Call to action
- 28:00Q&A
related talks
29mcudf.pandas: The Zero Code Change GPU Accelerator for Pandas
29mA Beginners Guide to GPUs for Pythonistas
31mcuDF: RAPIDS GPU-Accelerated Dataframe Library
01h 12mGPU-Accelerated Data Analytics in Python
01h 36mRAPIDS: GPU-Accelerated Data Science for PyData Users
02h 28mGPU Accelerated Python
39mAccelerating Data Science with RAPIDS
01h 22mHigh-Performance Data Science at Scale with RAPIDS, Dask, and GPUs