Ensemble Techniques for High Performance Machine Learning
summary
Ensemble stacking combines diverse model families—logistic regression, SVM, neural nets, gradient boosting, extra trees and AdaBoost—using cross-validated out‑of‑fold predictions and a mix of weighted and geometric averaging; diversity of features, targets and training algorithms drives gains. A production‑style Kaggle solution trains 33 level-one models and 3 level-two models, uses fixed fold indexes and OOF predictions, and reports ~0.396 CV logloss with a leaderboard around 0.38, demonstrating clear uplift at the cost of heavier training and maintenance.
AI-written summary · about
chapters (13)
- 0:00Introduction and speaker background
- 3:10What is ensemble stacking
- 8:25Why stacking works diversity
- 13:10Improving diversity feature engineering
- 16:28Diversity through training algorithms
- 18:37Target transformation for regression
- 20:04Stacking architectures explained
- 23:41Stacking workflow and cross-validation
- 26:59Tips to improve stacking performance
- 31:21Kaggle competition insights
- 36:00Case study Auto Group competition
- 39:14Advantages and disadvantages of stacking
- 41:45Q&A session
related talks
20mMulti-algorithm Ensemble Learning at Scale: Software...
40mBalancing scale and interpretability in analytical applications with sklearn and e
41mWhat Kaggle has learned from 2MM machine learning models
31mEnsemble X Your personal strataGEM to build Ensemble
30mMachine Learning Techniques for Class Imbalances & Adversaries
30mScalable Automatic Machine Learning with H2O
37mAutomated Machine Learning: Building a Pipeline
11mTree Ensemble Classifiers on heterogeneous platforms