Evaluating Model Explainability Methods
summary
Paper evaluates pixel-level feature importance methods (gradients, integrated gradients, guided backprop) by forcing a controlled mean-shift invariance test where model behavior is unchanged yet many estimators flip, and by a remove-and-retrain experiment that deletes top-ranked pixels then retrains — most estimators perform no better than random. An empirical fix—averaging ~15 noisy estimates and squaring then averaging—substantially improves reliability. LIME isn’t evaluated here, code and benchmarks are being open sourced for others to test.
AI-written summary · about
chapters (15)
- 0:00Intro
- 2:33Saras background
- 6:27Research vs applied
- 11:41Google Brain
- 15:20Datasets
- 20:56Misconceptions
- 27:03Interpretability vs Explainability
- 28:55Saras work
- 36:37Ranking estimators
- 42:07Evaluating feature importance estimates
- 44:54Frustration with diffuse methods
- 47:10Specific approach to ensemble
- 49:24Deep Learning in Africa
- 52:40Saras work in model compression
- 57:37Saras research landscape
related talks
48mInherent Trade-offs with the Local Explanations Paradigm
09mQuantitative Evaluation of Machine Learning Explanations: A Human-Grounded Benchmark
34mInterpreting Machine Learning Models: Why and How!
24mWhy and How to Build Explainability into your ML Workflow
16mBuilding Trust through Explainable AI
01h 29mNavigating ML Explainability: From Theory to Implementation
30mTowards a more transparent AI - Decrypting AI models using LIME
45mExplainable Machine Learning Explained