Deep Dive into scikit-learn's HistGradientBoosting Classifier..

on YouTube ↗32m

" summary

scikit-learn implements histogram GBDT that claims comparable accuracy and training time to XGBoost CatBoost and LightGBM on large problems by using histogram binning with 255 bins, Cython kernels and Cython + OpenMP parallelism; benchmarks on the Higgs dataset (reported 88 million rows and 28 features) run on 12 cores and show similar ROC and times (tens of seconds to minutes, ~85s for 100 rounds on a laptop). native missing-values support is implemented in the dev build and the approach works in practice.

AI-written summary · about

" chapters (12)

1. 0:00scikit-learn and boosting intro
2. 4:00Histogram gradient boosting
3. 8:58Building histograms for splits
4. 12:35HistGradientBoosting in scikit-learn
5. 16:54Parallelizing with OpenMP
6. 17:15Key hyperparameters
7. 20:48Early stopping and tree growth
8. 22:17Benchmarking libraries
9. 24:31Live demo: verbose mode
10. 25:21Missing values and roadmap
11. 26:10Install and use classifier
12. 27:41Q&A: binning trade-offs

" related talks

• 31mDo You Want To Build A Forest?Thomas J Fan · PyData Austin 2019
• 18mCompetitive Analysis of the Top Gradient...Sai Ayachit, Shyam R, Anubhav Singh, Vinayak Patil · OpenPOWER Summit NA 2020
• 38mCatBoost - the new generation of gradient boostingAnna Veronika Dorogush · PyData London 2018
• 01h 33mCrunching Your Data with CatBoost - the New Gradient Boosting LibraryVasily Ershov · PyData NYC 2018
• 01h 28mMastering gradient boosting with CatBoostAnna Veronika Dorogush · PyData London 2019
• 39mPushing Cython to its Limits in Scikit-learnThomas J Fan · PyData NYC 2024
• 37mUnlocking the Power of Gradient-Boosted Trees (using LightGBM)Pedro Tabacof · PyData London 2022
• 40mGradient Boosting Machines (GBM): from Zero to Hero (with R and Python code)Szilard Pafka · Data Con LA 2020