Evaluating fairness in machine learning with PyMC3
summary
Claims a concrete probabilistic threshold model for stop‑and‑search, implemented first in PyMC3 then PyStan after hitting a modeling limitation, and fits UK stop‑and‑search records to infer latent stop thresholds and hit rates. Results show a large delta in stop rates by ethnicity and lower hit rates for some groups, posterior distributions are wide and overlapping so conclusions remain tentative, and a formal impossibility result forces trade‑offs among fairness metrics so the recommendation is: measure many metrics and choose policy tradeoffs.
AI-written summary · about
chapters (11)
- 0:00Introduction to fairness in machine learning
- 3:19Biases in data collection
- 9:06Fairness through unawareness
- 13:21Demographic parity limits
- 17:09Error rate fairness
- 21:26Threshold adjustments
- 23:55Impossibility condition
- 25:45Measuring fairness in data
- 31:01Probabilistic stop and search
- 35:31Trade-offs in fairness
- 36:36Q&A
related talks
20mCamelia
42mFairness and transparency in machine learning: Tools and techniques
28mBeyond Prediction Performance: How Modeling Decisions Shape Fairness Outcomes in Statistical...
42mFairness in Machine Learning: From Theory to Practice
46mThe Measure and Mismeasure of Fairness
28mMeasuring Model Fairness
39mBeyond Bias: Crafting Fairness in Pythonic Machine Learning Systems
50mThe Measure and Mismeasure of Fairness