Frequency and Bayesian Approaches to Estimating TPR with Partial Labels
summary
Compares frequency-based and Bayesian methods for estimating true positive rate (TPR/Recall) when labeled data is incomplete; discusses practical evaluation of online ML model metrics like Precision and Recall and shows how the two statistical approaches handle partial labeling. Examples and techniques focus on estimating TPR under missing or imperfect labels, including how to apply frequency estimates and Bayesian inference to produce more reliable online metric measurements.
AI-written summary · about
chapters (6)
- 0:00Introduction
- 3:20Incomplete label problem
- 7:15Frequency approach
- 15:53Bayesian approach
- 26:36Comparing approaches
- 29:07Q&A
related talks
27mI have no clue what my model is doing...
35mProbably Approximately Precision and Recall Learning
22mHow to Detect Silent Failures in ML Models
34mPrediction intervals
31mBias Meets Bayes: A Bayesian Perspective on Improving Model Fairness
33mWeak supervision: a new paradigm for unreliable data
23mHow to detect silent failures in ML models
40mModel Upgrade Schemes- Considerations for Updating Production Models