Better Than Deep Learning: Gradient Boosting Machines (GBM)
summary
Gradient boosting routinely outperforms neural nets on relational tabular data: use xgboost, lightgbm, h2o and expect to tune learning rate, tree depth and number of trees with early stopping; benchmarks show large speed and memory differences — XGBoost uses ~6GB GPU RAM on 100M rows while Spark is ~200× slower and uses ~100× RAM for similar jobs. Workflows that win use heavy feature engineering, random hyperparameter search, and ensembles of dozens–hundreds of GBM models.
AI-written summary · about
chapters (19)
- 0:00Introduction and speaker background
- 1:12Deep learning myths and realities
- 4:05Why gradient boosting outperforms deep learning
- 10:24How gradient boosting works
- 12:13Top open-source GBM tools
- 14:47Big data and machine learning feasibility
- 17:22Speed vs. data size in machine learning
- 18:06Benchmarking GBM implementations
- 21:32Deployment and real-time scoring
- 22:55Spark’s limitations for GBM
- 25:52Distributed machine learning challenges
- 27:17GPUs and GBM performance
- 28:01Key GBM parameters explained
- 30:10Tuning gradient boosting models
- 31:16CPU parallelization and NUMA effects
- 34:49Parallelizing hundreds of models
- 35:32Choosing the right GBM tool
- 37:18Winning Kaggle strategies
- 38:25Q&A session
related talks
40mGradient Boosting Machines (GBM): from Zero to Hero (with R and Python code)
37mUnlocking the Power of Gradient-Boosted Trees (using LightGBM)
38mCatBoost - the new generation of gradient boosting
18mCompetitive Analysis of the Top Gradient...
42mDeep Learning is Like Water (it's Everywhere!)
30mUnderstanding GPUs
32mDeep Dive into scikit-learn's HistGradientBoosting Classifier..
01h 01mINTERPRETABLE AND DIFFERENTIALLY PRIVATE MACHINE LEARNING