Dehumanizing Agents: Why Explainability is Crucial in the LLM Era
summary
Argues that explainability is nonnegotiable for LLMs and shows concrete tools: LIME proves fragile on text and scales poorly, so an internal AutoML called auto explain picks only transparent models and scores them on compactness and stability (worked and is reproducible from the paper). For generative LLMs, RAG plus ablation gives practical source attribution but needs a trained scope detection classifier (they use a three‑class model in production). Also uses prompt chaining, chain/tree‑of‑thoughts and notes OpenAI’s confessions research is useful but not public.
AI-written summary · about
chapters (23)
- 0:00Introduction
- 1:20What is explainable AI
- 4:52Clients resisted explainability
- 6:39EU AI Act requirements
- 7:21Global vs local explainability
- 10:11Implementing explainability challenges
- 10:55AutoExplain framework
- 13:48Limitations of interpretable AI
- 14:08How LLMs function
- 16:18LLM explainability challenges
- 17:01Humanizing LLMs risks
- 19:06Dangers of overtrusting LLMs
- 21:10Explainability techniques for LLMs
- 25:49RAG and source attribution
- 29:08Validating source attribution
- 33:25Third-party LLM services citations
- 35:37Asking LLMs for explanations
- 37:02Prompt chaining for transparency
- 38:26Chain of thought techniques
- 41:15Claude thinking mode limits
- 43:03OpenAI's confessions mechanism
- 46:14Human validation limits
- 47:26Final thoughts & Q&A
related talks
56mDehumanizing Agents: Why Explainability is Crucial in the LLM Era
28mExplaining Machine Learning Models with LLMs
35mWhy Do You Need an Explainable Ai?
01h 29mNavigating ML Explainability: From Theory to Implementation
24mWhy and How to Build Explainability into your ML Workflow
46mAI for High-Stakes Decision Making
16mBuilding Trust through Explainable AI
28mManaging AI Risk: Beyond Explainability