How ChatGPT Works
summary
Explains that ChatGPT is a multi‑billion‑parameter transformer that turns tokens into vectors, uses attention heads to weight context, and predicts the next token via backprop training on massive text corpora; shows word embeddings (king − man + woman ≈ queen), demos of generated code and a rap, and notes fine‑tuning by humans as fine‑tuned with RLHF to reduce harm; claims emergent abilities appear once scale crosses thresholds and that alignment still fails in surprising ways.
AI-written summary · about
chapters (10)
- 0:00GPT and surprises
- 1:26Neural networks approximate functions
- 8:37Neural networks: layers and training
- 15:48Word embeddings and language models
- 23:24Training GPT to predict next word
- 33:27Attention mechanisms in GPT
- 41:25Fine-tuning language models
- 46:25Emergent abilities of LLMs
- 50:43Philosophy and future of GPT
- 51:27Question and answer session
related talks
25mEmbeddings, Transformers, RLHF: Three key ideas to understand ChatGPT
46mConversations with ChatGPT: illusion or reality?
01h 01mKeynote: Transformers: The Rise of ChatGPT
35mChatGPT is lying, how can we fix it?
01minteresting experience with ChatGPT
01h 16mChatGPT - and How we Can Benefit from it
02h 00mChatGPT & Prompt Engineering
01h 33mThe Developer's Guide to ChatGPT API