Cost-effective data annotation with Bayesian experimental design
summary
Claims that Bayesian experimental design picks labels far more efficiently than random sampling, shows a concrete 100-element array example where the optimal query lands in the middle, and proves binary search is a special case of the BED objective. Shows Python code and a usable package called opt-based-experiment library, reports lower posterior entropy and lower error in simulations versus naïve strategies, and walks through the exact expected-entropy computation used to choose each next label.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 1:06Experimental design examples
- 4:35Modeling data generation
- 8:51Bayesian design loop
- 15:37Bayesian model and posterior
- 19:35Uncertainty with Shannon entropy
- 20:39Binary search as design
- 26:57Disease modeling application
- 29:03Names and applications
- 30:29Resources and closing
related talks
01h 30mBayBE: A Bayesian Back End for Experimental Planning in the Low-To-No-Data Regime
25mScaling Bayesian Experimentation
39mModern Bayesian Experimental Design
28mBayesian Optimization- Fundamentals, Implementation, and Practice
34mBayesian Optimization and it's application to Neural Networks
01h 04mIntroduction to Experimental Design
41mKeynote: Personalization, Bandits, and Causal inference
38mUsable A/B testing – A Bayesian approach