Efficient and Flexible Hyperparameter Optimization
summary
BOHB claims to combine Hyperband’s multi‑resolution budgeted evaluations with a Bayesian surrogate (tree‑parzen style) to get fast and robust hyperparameter search, and the Freiburg group ships it as BOHB implemented in hpbandster. The implementation supports conditional and mixed-type spaces, parallel multi‑node workers, and switches to a model once enough low‑budget results exist, producing reported gains like a 55x speed‑up over random search in benchmarks.
AI-written summary · about
chapters (14)
- 0:00Hyperparameter optimization intro
- 1:24Why HPO matters
- 3:32Manual vs algorithmic tuning
- 5:22Formal HPO problem
- 6:46HPO algorithm requirements
- 8:58Model-free search methods
- 11:25Bayesian optimization
- 14:14Successive halving & Hyperband
- 17:02Combining Bayesian optimization with Hyperband
- 19:52HPBandSter library
- 22:02Flexible config spaces
- 23:27Other HPO & AutoML libs
- 24:10Multi-resolution evaluation
- 24:45Q&A
related talks
38mA Guide to Modern Hyperparameters Turning Algorithms
46mBayesian Optimization Accelerator
18mBest Practices for Hyperparameter Optimization
24mUsing Bayesian Optimization to Tune Machine Learning Models
28mBayesian Optimization- Fundamentals, Implementation, and Practice
45mBest Of Both Worlds: Stochastic & Adversarial Best-Arm Identification
01h 30mBayBE: A Bayesian Back End for Experimental Planning in the Low-To-No-Data Regime
16mHydro: Surrogate-Based Hyperparameter Tuning Service in Datacenters