Conformal Prediction: uncertainty quantification to humanise models
summary
Conformal prediction turns point outputs into prediction sets with mathematical coverage by computing nonconformity scores on a reserved calibration set and selecting a quantile q̂ based on an error rate α so coverage ≈ 1−α under exchangeability. It is model‑agnostic and implemented with tools like MAPIE and SCRAPS; extensions include MBPI, adaptive conformal inference and conformalized quantile regression. At INGV a CatBoost + conformalized quantile regression pipeline forecasts GNSS amplitude scintillation in real time, and online updates to nonconformity scores recover coverage after abrupt change points.
AI-written summary · about
chapters (11)
- 0:00Conformal prediction intro
- 1:16Why uncertainty matters in machine learning
- 3:48Calibration vs conformal
- 6:17How conformal works
- 8:25Demo: image classification
- 10:34Human-in-the-loop pipelines
- 13:30Ethics in human-centered AI
- 14:34Conformal for time series
- 19:35Financial risk management
- 20:38Space weather forecasting
- 23:15Q&A on guarantees
related talks
31mUncertainty quantification: How much can you trust your machine learning model?
36mQuantifying uncertainty in Machine Learning predictions
19mUnderstanding, Generating, and Evaluating Prediction Intervals
32mLeveraging Conformal Prediction for Calibrated Probabilistic Time Series Forecasts
22mConformal Inference with Tidymodels
35mUncertainty Unleashed
49mBoosting AI Reliability: Uncertainty Quantification with MAPIE
30mBoosting AI Reliability- Uncertainty Quantification with MAPIE