Explainable AI as a Conventional Data Analysis Tool
summary
Claims that explainable ML becomes a conventional analysis step by extracting a SHAP values matrix from a trained model (examples use XGBoost and the shap library), clustering those instance-level importances with k-means clustering and Calinski‑Harabasz to pick cluster count, then running regressions per cluster. Applied to credit default (and demos on Titanic and readmission data) this raises fit metrics notably — R² 0.18 → 0.36 and accuracy 0.84 → 0.87 — and surfaces unexpected predictors like whether an email or work phone was provided.
AI-written summary · about
chapters (12)
- 0:00Explainable AI introduction
- 3:41Accuracy versus explainability
- 6:10Rising explainable AI research
- 7:55Explainable AI techniques
- 10:03SHAP and Shapley values
- 14:20SHAP data transformations
- 16:31SHAP with regression clustering
- 18:39Credit default case study
- 25:29SHAP Python demo
- 31:44Q&A on SHAP
- 35:41Using explainers beyond model interpretation
- 38:37Closing summary
related talks
24mWhy and How to Build Explainability into your ML Workflow
27mInterpretability and explainability in machine learning
31mBeyond the Black Box: Interpreting ML Models with SHAP
01h 29mNavigating ML Explainability: From Theory to Implementation
35mTaking a peek under the hood - Interpreting black box model
15mIntroduction to Explainable AI
35mSHAPtivating Insights: unravelling blackbox AI models
47mML: Explainable Artificial Intelligence explained