Optimizing Machine Learning Models
summary
Bayesian global optimization via SigOpt claims to find better hyperparameters far faster than grid or random search, trading exploration and exploitation using Gaussian processes and expected improvement. It bolts onto TensorFlow, XGBoost and custom trading strategies; on Lending Club XGBoost it reaches top accuracy with orders-of-magnitude fewer trials, on a Quantopian trading backtest it beats expert tuning by ~200% and runs ~10× faster, on CNNs it cuts published error rates (1.6% and up to 15% relative). Typical scale: few dozen parameters, 100–300 suggestions, 100× faster for 5–10D problems.
AI-written summary · about
chapters (6)
- 0:00Introduction and motivation
- 10:39Real-world tuning examples
- 20:08Bayesian optimization math
- 26:25Deep learning tuning
- 33:11Neural network case studies
- 39:39Data scale and pricing
related talks
24mUsing Bayesian Optimization to Tune Machine Learning Models
18mAlexandra Johnson, Software Engineer, SigOpt, NYC 2017
18mBest Practices for Hyperparameter Optimization
26mAddressing Some of Challenges When Optimizing Long-to-Train-Models
34mBayesian Optimization and it's application to Neural Networks
28mBayesian Optimization- Fundamentals, Implementation, and Practice
26mScott Clark, CEO & Co-Founder, SigOpt at MLconf Seattle 2017
39mBayesian optimisation with scikit-learn