Adventures in scikit-learn's Random Forest
summary
Uses scikit-learn RandomForestRegressor on an ASX equities dataset, cleans NaNs with median imputation and replaces all-NaN features with zeros, and constrains trees via n_estimators and max_features (plus depth/sample-leaf limits). Uses out-of-bag CV instead of full cross-validation. Resulting r^2 is negative (−0.02) while rank correlations are modest (Kendall ≈ 11%, Spearman ≈ 15%). A backtest shows the model outperforms the incumbent in 2014 but underperforms in 2013, so the build works only intermittently.
AI-written summary · about
chapters (21)
- 0:00Intro
- 1:01A Brief Introduction to Supervised Learning
- 1:15Classification
- 1:37Regression
- 1:56Training Data
- 2:46Test Data
- 4:25Why Random Forests?
- 8:20Decision Trees
- 9:29Random Sample with Replacement
- 11:14Number of Trees
- 11:54Number of Features
- 13:18Depth of Trees
- 14:59Installation
- 15:32Imputation
- 19:29Missing Features
- 25:20Evaluating Results
- 27:38Alternatives
- 29:30Cross-validation
- 30:32Out-of-bag Estimates
- 34:40Accessing the Trees
- 35:11More scikit-learn and Random Forests
related talks
27mUsing Random Forests in Python
44mUnderstanding Random Forests
30mFeature Engineering: An Apprentice’s Guide to the “Dark Art” of Machine Learning
33mQuantifying uncertainty in machine learning models
21mWhy a 1990s Machine Learning Algorithm Destroys LLMs at...
54mDebiasing Random Forests For Treatment Effect Estimation
57mFrom Bonsais to Random Forests
53mRobust Algorithms for Machine Learning