Decision Making under Miscalibration
summary
Defines a minimax threshold ĵ that minimizes worst-case regret under a bounded miscalibration α and shows closed-form answers for both expected and max calibration error measures. Proof reduces the adversary to a two-point worst-case predictor (support on ≤2 scores), producing explicit formulas. Finds ĵ = j* for moderate cost asymmetry but shifts to conservative extremes near edges, so slight miscalibration can change whether treating everyone or no one is optimal. The construction is analytic and the derived thresholds are computable and tight.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 0:19Predictions to Decisions
- 1:34Utilitybased Framework
- 5:17Decision Making
- 7:45Risk Scores
- 9:49Optimal Threshold
- 10:53Solution Concept
- 16:03Visual Analysis
- 17:11Choice of Worst Case Regret
- 19:12Discussion Point
related talks
45mAgreement and Alignment for Human-AI Collaborative Decision Making
48mIncentivizing Exploration among Behavioral Agents with Unbiased Histories
43mMetric Learning for Individual Fairness
31mAccuracy and Interpretability Through the Lens of Human-AI Teaming
31mDecision theoretic foundations for human-AI collaboration
56mDecision Making in the Abscence of Ground Truth
41mFostering Trust via Explainable ML Inferences
22mInterface Design Optimization as a Multi-Armed Bandit Problem