Mechanistic Interpretability
summary
Mechanistic interpretability shows that transformers encode identifiable concept vectors rather than single neurons, found by supervised probes or heavy compute unsupervised methods like sparse autoencoders. Open-model tooling (NeuronPedia, Hugging Face repos, Transformers hooks) locates and amplifies a concept vector; an open LLaMA variant with Eiffel Tower steering shifts answers toward tower-related content and then breaks into gibberish as stimulation grows. The workflow is reproducible with a few lines of code and requires tuning amplitude and layer targets.
AI-written summary · about
chapters (11)
- 0:00Mechanistic interpretability intro
- 1:35Neural networks as black boxes and their risks
- 4:44Layer-wise feature detection
- 7:33Transformers and LLM interpretability
- 8:57Why neurons are hard to interpret
- 10:18Finding concept vectors
- 12:48Discovering circuits in LLMs
- 13:53Deep brain stimulation
- 15:43Concept amplification experiments
- 17:51Steering in LLMs
- 18:54Future directions
related talks
01h 01mBetween the Layers– Interpreting Large Language Models
01h 16mHow Do Transformers Work?
59mBetween the Layers– Interpreting Large Language Models
21mWhy you should care about AI interpretability
35mReading the Mind of an LLM
37mLinear representations of concepts in modern AI models
46mLanguage Models as World Models?
40mAn Occam's Razor Principle for Transformers?