Gradient Boosting for Data with Both Numerical and Text Features

on YouTube ↗37m

" summary

CatBoost claims automatic, high-quality handling of categorical and text features: target-leakage-free label encoding via time-ordered leave‑one‑out estimates, greedy on‑the‑fly categorical combinations, and a naive Bayes text projector for bag‑of‑words features. Open‑sourced library and GPU-enabled training show empirical wins over XGBoost/LightGBM on categorical datasets, GPU training reports up to ~40x speedup, and optimized C++ inference yields 20–50x faster prediction in some cases.

AI-written summary · about

" chapters (11)

1. 0:00Introduction
2. 1:41Gradient boosting basics
3. 8:44Categorical features handling
4. 11:58Why categorical handling
5. 16:57Avoiding target leakage
6. 20:10Text data in CatBoost
7. 23:52CatBoost features
8. 27:25Model analysis tools
9. 29:13Benchmarks and GPU
10. 33:13Summary
11. 34:01Q&A

" related talks

• 01h 27mCatBoost - The New Generation of Gradient BoostingVasily Ershov · PyData Los Angeles 2018
• 38mCatBoost - the new generation of gradient boostingAnna Veronika Dorogush · PyData London 2018
• 01h 28mMastering gradient boosting with CatBoostAnna Veronika Dorogush · PyData London 2019
• 01h 33mCrunching Your Data with CatBoost - the New Gradient Boosting LibraryVasily Ershov · PyData NYC 2018
• 40mCatBoost: Fast Open-Source Gradient Boosting Library For GPUVasily Ershov · PyData Berlin 2018
• 42mHow am I going to deal with all of these catsLiam Kirwin · PyData London 2019
• 18mCompetitive Analysis of the Top Gradient...Sai Ayachit, Shyam R, Anubhav Singh, Vinayak Patil · OpenPOWER Summit NA 2020
• 40mGradient Boosting Machines (GBM): from Zero to Hero (with R and Python code)Szilard Pafka · Data Con LA 2020