Mastering gradient boosting with CatBoost

on YouTube ↗01h 28m

" summary

CatBoost demonstrates concrete wins: out‑of‑the‑box categorical feature support, defaults that give strong quality without heavy tuning, and benchmarks where CPU performance varies but GPU speed‑up up to 50x (multiclass up to 100x) is routine. The tutorial uses Python/Jupyter, Pool objects, cross‑validation, grid search, early stopping/snapshotting, and SHAP for explanations. Prediction latency is a selling point — prediction 30–60× faster — and a GPU prediction layer for self‑driving use cases is implemented and fast.

AI-written summary · about

" chapters (42)

1. 0:00Introduction
2. 1:49Intro to CatBoost
3. 2:08Overview of the Presentation
4. 2:39Intro to Gradient Boosting
5. 6:08Numerical and Categorical Data with CatBoost
6. 7:26Advantages of CatBoost
7. 9:00Library Comparison (Quality)
8. 9:45Speed
9. 10:11Benchmarking (CPU & GPU)
10. 11:55CPU vs GPU
11. 12:50Prediction Time
12. 13:24Tutorial
13. 15:15Problem Statement
14. 15:38CatBoost Library (Imports and related issues)
15. 16:22Reading and Intro to the Data
16. 18:17Exploring the data
17. 19:36Training the Model with default parameters
18. 22:16Creating the Pool Object
19. 23:12Splitting the data (Train & Validation)
20. 24:16Selecting the objective function
21. 25:11STDOUT of training
22. 28:32Plotting metrics while training
23. 30:33Model Comparison (plotting after training)
24. 32:39Finding the best model
25. 35:05Cross-Validation
26. 41:30Grid Search
27. 44:40Overfitting Detector
28. 49:18Overfitting Detector with eval metric
29. 51:31Model Predictions
30. 57:10Select Decision Boundary
31. 1:01:04Model Evaluation (new dataset)
32. 1:03:06Feature Importance
33. 1:03:37Prediction Values Change
34. 1:04:50Loss Function Change
35. 1:07:49Shap Values
36. 1:16:05Snapshotting
37. 1:17:45Saving the Model
38. 1:18:36Hyperparameters Tuning
39. 1:23:07Speeding up Training and Reducing Model Size
40. 1:23:35Additional Details about CatBoost Community
41. 1:25:50Future Scope of CatBoost
42. 1:26:22Questions and Suggestions

" related talks

• 01h 27mCatBoost - The New Generation of Gradient BoostingVasily Ershov · PyData Los Angeles 2018
• 01h 33mCrunching Your Data with CatBoost - the New Gradient Boosting LibraryVasily Ershov · PyData NYC 2018
• 40mCatBoost: Fast Open-Source Gradient Boosting Library For GPUVasily Ershov · PyData Berlin 2018
• 38mCatBoost - the new generation of gradient boostingAnna Veronika Dorogush · PyData London 2018
• 37mGradient Boosting for Data with Both Numerical and Text FeaturesVasily Ershov · PyData Los Angeles 2019
• 18mCompetitive Analysis of the Top Gradient...Sai Ayachit, Shyam R, Anubhav Singh, Vinayak Patil · OpenPOWER Summit NA 2020
• 16mStop Failures Before They Happen: Gradient BoostingPavel Solikov · Conf42 Machine Learning 2026
• 40mGradient Boosting Machines (GBM): from Zero to Hero (with R and Python code)Szilard Pafka · Data Con LA 2020