Fast Confidence Estimation: Regression models
summary
Claims a p-value based confidence metric that judges counts or ratios without ground truth by mapping them to a fitted CDF (normal or spline) and returning doubled-minimal tail probability. Implements numeric CDF via trapezoid rule, exposes a compile-time CDF lookup table generated in C++17 to avoid runtime cost, and provides a custom compile-time exp via Taylor plus fast integer power. Results are constant-time lookups with low startup overhead and practical use for profile guided optimization on Android constant-time queries and verifiable outputs on GitHub.
AI-written summary · about
chapters (11)
- 0:00Introduction and Overview
- 0:13Understanding Prediction Confidence
- 1:14Challenges in Prediction Validation
- 1:41Profile Guided Optimization Method
- 3:56Defining Confidence Metrics
- 5:01Evaluating Confidence Metrics
- 10:29Implementation Details
- 11:11Efficient Computation Techniques
- 12:41Generating Lookup Tables
- 17:30Advanced Computation Methods
- 20:08Conclusion and Summary
related talks
45mUncertainty estimation at scale with functime
17mCCEval: Accurately and Confidently Evaluating Performance Metrics of Congestion...
36mQuantifying uncertainty in Machine Learning predictions
31mUncertainty quantification: How much can you trust your machine learning model?
35mUncertainty Unleashed
37mQuantifying Uncertainty in Time Series Forecasting
34mPrediction intervals
30mStatistical Property Testing and Estimation Beyond the i.i.d. Setting