Reading the Mind of an LLM
summary
Claims that mechanistic interpretability can locate and manipulate feature directions inside real LLMs — using sparse dictionary learning, circuit tracing and attribution graphs — and shows results on models with hundreds of millions to hundreds of billions of parameters. Labs report isolating phenomena like emotion vectors and a persistent “golden gate” concept, and demos steer persona via steering vectors derived from residual activations. Teams train larger sparse surrogate layers with L1 penalties to recover features; methods work at scale but feature labels remain subjective and not fully causal.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 4:44Residual stream and meaning
- 10:40Activations and linear algebra
- 14:51Superposition and interference
- 23:57Linear representation hypothesis
- 28:11Golden Gate Claude feature recovery
- 32:01Circuit mapping and steering
- 34:10Takeaways and future of interpretability
related talks
28mExplaining Machine Learning Models with LLMs
52mGeneralization in the representations and computations of frontier language models.
59mBetween the Layers– Interpreting Large Language Models
50mDehumanizing Agents: Why Explainability is Crucial in the LLM Era
01h 01mBetween the Layers– Interpreting Large Language Models
37mLinear representations of concepts in modern AI models
44mThe LLM Revolution: From Inception to the Future of Thinking Machines
41mEnhancing LLMs with Graph Technology