3 Ways to Speed up Your Regression Modeling in Python
summary
Benchmarks show a 20 million–row regression with 60 one‑hot features runs ~10s with a naive normal‑equation, speeds ~30% using sparse matrices (scikit‑learn / scipy), drops to ~2.7s using the Frisch‑Waugh‑Lovell theorem (pyfixest) and to ~4s with out‑of‑memory aggregation via DuckDB compression, which reduces the data to ~2k unique strata and fits lossless weighted least squares returning matching coefficients and recoverable standard errors; JAX/GPU demeaning experiments also give further speedups.
AI-written summary · about
chapters (13)
- 0:00Regression speed challenges
- 1:30Use case and dataset
- 4:02Baseline model
- 5:49Large model challenges
- 7:15Speed-up strategies
- 8:40Sparse data structures
- 11:07Batch regression
- 14:44Frisch-Waugh-Lovell theorem
- 18:19Data compression
- 27:57Out-of-memory regression
- 28:19Benchmarking summary
- 31:08GPU acceleration
- 31:51Q&A
related talks
21mKeynote: Easier Machine Learning Thoughts From Scikit-Learn
40mTools and Tricks from a Pragmatic Data Scientist
24mVectorise all the things! How basic linear algebra can speed up your data science code
40mA Practical Guide to Dimensionality Reduction Techniques
44mActive Regression via Linear-Sample Sparsification
27mSpeed up Python with hardware accelerators
28mSketching for M-Estimators: A Unified Approach to Robust Regression
30mBreaking the Speed Limit: Fast Statistical Models with Python 3.14...