Bayesian Optimization and Machine Learning for Accelerating Experiments in the Physical Sciences
summary
Describes a practical pipeline that recalibrates confidence intervals for regression and classification by learning a mapping from predicted probabilities to empirical frequencies on held-out data (isotonic or simple regressors). Shows that calibrated models improve model-based RL and decision making, and that coupling a racing algorithm with predictive early stopping (linear predictor on handcrafted time‑series features) finds the best charging protocol among 225 candidates in 16 days versus ~1.5 years of full tests. Also reports 4x faster tuning of SLAC free electron laser magnets; methods are implemented and validated on real experiments.
AI-written summary · about
chapters (25)
- 0:00Intro
- 4:40Ingredients for Scientific Discovery
- 7:41Today: Data-Driven Scientific Discovery
- 10:26Why Model Uncertainty?
- 12:01Machine Learning Systems Minimize Training Loss
- 25:07Recalibration for regression
- 31:02Calibration for decision making
- 32:19Depth Estimation
- 32:53Calibration improved across the board
- 34:11Model-based reinforcement learning
- 40:38Today's Talk: Experiment Design
- 41:04Experiments Are Costly
- 42:11Challenge: Charging Time
- 43:01Fast charging protocol optimization Multistage Constant Current
- 45:35Learning Fast Charging Protocols
- 46:55Cost of Battery Experiments
- 48:04Optimal Experimental Design
- 48:30Example: Racing Algorithms
- 49:08Race Ahead With Confidence
- 50:42Opening the Experiment Blackbox
- 50:54Real Sim Experiments
- 51:29Real + Sim Experiment Design (RSED)
- 51:54Proposed Algorithm: Racing with RSED
- 54:31Empirics: Battery Fast Charging
- 56:17Summary: Real + Sim Experiment Design
related talks
44mHandling predictive uncertainty in Machine Learning
01h 30mBayBE: A Bayesian Back End for Experimental Planning in the Low-To-No-Data Regime
28mBayesian Optimization- Fundamentals, Implementation, and Practice
24mUsing Bayesian Optimization to Tune Machine Learning Models
41mOptimizing Machine Learning Models
25mScaling Bayesian Experimentation
46mBayesian Optimization Accelerator
34mBayesian Optimization and it's application to Neural Networks