AI Bias Testing and Mitigation: How to Choose the Right Open Source Tools
summary
Compares three named tools and says which to use: Audit AI = quick pass/fail gives disparate impact and threshold plots with minimal code; AIF 360 for production audits provides a full toolkit, explainer text for every metric and easy automation for many models; What‑If Tool for interactive feature inspection surfaces outliers and confounders in a notebook. Recommends always checking disparate impact, choosing tests that target false positive or false negative harms, and notes mitigation often means changing labels, cutoffs, or the modeling approach rather than a plug‑in fix.
AI-written summary · about
chapters (23)
- 0:00Introduction
- 1:40Agenda
- 2:10Software is a tool
- 3:18Bias
- 5:22What is Bias
- 8:10What is Bias Testing
- 11:20False Negative Rate and False Positive Rate
- 13:39Bias Testing Tools Overview
- 17:55Are Bias Tests the Same
- 19:07AI F3
- 20:29AI 360
- 21:21Metrics
- 24:15WitWidget
- 26:28Health vs Cost
- 27:30Whip Widget
- 29:19Widget config builder
- 30:04Bias mitigation
- 30:41Bias detection
- 31:30Bias for people of color
- 35:05AI360
- 35:28Training
- 36:57Docs
- 37:11Questions
related talks
47mFight AI Bias With… Bias
43mAI for Testing – Panel Discussion
54mAI: Actual Intelligence
53mIt’s Not Fair! Detecting Algorithmic Bias with Open Source Tools
14mBuilding with the Responsible AI toolkit
56mAn AI with an Agenda: How Our Cognitive Biases Leak Into Machine Learning
49mAn AI with an Agenda
23mRemoving Unfair Bias in the Machine Learning Pipeline