What Kaggle has learned from 2MM machine learning models
summary
Kaggle reports 2 million models submitted and now sees over 100,000 model submissions per month, each taking roughly 5–10 hours to develop, which yields concrete lessons: two strategies dominate — XGBoost vs deep nets (hand‑crafted features + gradient boosting for structured data, CNNs for images, RNNs for sequences) — ensembles of uncorrelated top models improve scores, correlation maps reveal when a dataset’s signal is exhausted, surprising wins include a glaciologist beating NASA, and the new Kaggle Scripts notebook/forging environment makes code reproducible and widely reused.
AI-written summary · about
chapters (19)
- 0:00Introduction
- 1:36What is Kaggle
- 2:28Kaggle community
- 8:21Lessons from competitions
- 10:24Martin OLeary
- 15:33Competitions
- 16:27Correlation maps
- 18:48Motion with accelerated speed
- 21:19Ensemble
- 22:24Heart failure
- 23:29Correlation map
- 24:08Hand rolled feature
- 25:39Deep neural networks
- 26:43Recurrent neural networks
- 28:16Creativity
- 29:03Overfitting
- 30:52Version Control
- 32:33Getting Started Competition
- 38:25Competition Rules
related talks
21mBen Hamner, Co-founder and CTO, Kaggle
44mEnsemble Techniques for High Performance Machine Learning
12mCommunity matters: 8 reasons why you should be involved with
41mBetter Than Deep Learning: Gradient Boosting Machines (GBM)
37mRecent Solutions Using Deep Learning Models/ The How and Why
55mWhat Skills Do Predictive Modelers Need?
38mBuilding robust machine learning systems
37mAutomated Machine Learning: Building a Pipeline