Neural Networks
summary
Step-by-step math shows how a decoder-only Transformer actually converts tokens to numbers (word embeddings, positional encoding), computes dot-product similarities with masked self-attention, applies softmax and residuals, and uses a fully connected layer plus backprop to train weights. A tiny worked example generates the prompt continuation (it literally generated "awesome" then EOS), so the demo is concrete and reproducible. Ends with a concrete proposal to try a dopamine simulation idea to bring biology into future models.
AI-written summary · about
chapters (16)
- 0:00Neural networks & Transformers
- 1:27Decoder-only Transformers step by step
- 6:50Backpropagation trains networks
- 8:18Word embeddings to numbers
- 9:02Position encoding for word order
- 10:29Masked self-attention captures relationships
- 19:20Residual connections simplify training
- 20:44Fully connected layers generate output
- 22:52Transformer response generation
- 26:50Transformers vs biological networks
- 27:10Perceptrons & convolutional history
- 31:04Why Transformers broke modeling
- 32:06Physics inspiration in graphics
- 33:09Future neural network research
- 33:31Dopamine in learning
- 35:41Closing & Q&A
related talks
26mA Shallow Introduction to Self-Attention
01h 26mFrom RNNs to Transformers: A Journey through Language Modeling
28mEverything you need to know about Transformer Models
37mCode First, Math Later: Learning Neural Nets Through Implementation and Examples
01h 29mBuild Your Own Transformer
40mAn Occam's Razor Principle for Transformers?
01h 25mPaper Discussion: Attention is all you need
26mRethinking the Transformer: Toward Native Multimodal Architectures