Boosting AI Reliability: Uncertainty Quantification with MAPIE
summary
An open‑source Python tool MAPIE scikit‑learn‑contrib wraps estimators to output calibrated prediction intervals and sets with statistical guarantees (example: 95% coverage on test splits), supports split and cross‑conformal strategies, conformalized quantile regression (CQR) for heteroskedastic regression, jackknife/CV/APS/label/topk rules for classification, a Learn‑Then‑Test risk‑control pipeline for precision guarantees, and works with pretrained models (e.g. Hugging Face BERT); demos use ~600 points, recommend ≥200 for calibration.
AI-written summary · about
chapters (7)
- 0:00AI reliability intro
- 1:23Uncertainty quantification
- 8:44Conformal prediction basics
- 15:16MAPIE regression demo
- 45:36MAPIE classification
- 1:03:20Risk control with MAPIE
- 1:23:13Closing & Q&A
related talks
30mBoosting AI Reliability- Uncertainty Quantification with MAPIE
49mBoosting AI Reliability: Uncertainty Quantification with MAPIE
44mAccuracy Is Not Enough: Building Trustworthy AI with Conformal Prediction
30mConformal Prediction: uncertainty quantification to humanise models
31mUncertainty quantification: How much can you trust your machine learning model?
34mPrediction intervals
35mUncertainty Unleashed
44mHandling predictive uncertainty in Machine Learning