Use of Python for Cutting edge Language Model research
summary
Claims that model functions can be ported by copying an average activation vector across contexts, implemented with the NN-Sight Python API and a remote NDIF cluster (runs up to a 70B model). Code builds an intervention graph (runner/invoker), reads the final-token hidden state across a batch (layer 8 in examples), averages it, then adds it to zero-shot inputs; this shifting of logits “moves the needle” reproducibly on some prompts, and drill into attention heads amplifies the effect.
AI-written summary · about
chapters (16)
- 0:00intro
- 0:20preamble
- 0:40intro to mechanistic interpretability
- 2:06mech interp
- 2:57mech interp toolkit: causal interventions
- 5:10example mech interp research
- 16:59interpretability libraries and packages
- 17:44nnsight architecture
- 20:42example intervention
- 23:30anatomy of an intervention
- 27:43model internal i/o are nodes on the intervention graph
- 28:33information flow
- 29:11average-out information vectors
- 31:06getting the average activation vector
- 34:45adding the average to one-shot activation vector
- 39:15resources
related talks
44mAI Intuition: Exploring Language Model Latent Space
41mSub-Linear Algorithms meets Large Language Models
25mML model from an idea to production with the help...
30mAnyone can Build Great Deep Learning Applications - Deep Numpy
50mCreating a scalable (...) with Elixir, Python & Tensorflow
36mNeural Networks
48mDynamic Neural Networks with DyNet
30mAccelerating AI Application: NAVER's Journey from Model to Application