Inside GPT – Large Language Models Demystified
summary
Demonstrates that a language model is literally a next‑token predictor and shows a runnable demo: a GPT-2 on PyTorch chat bot delivered from a laptop using Hugging Face Transformers and Jupyter/VS Code, which works but produces plausible‑sounding errors (example: asserts lions in Oslo). Reveals concrete internals: ~50,257 token vocabulary, 768‑dim embeddings, 12 layers × 12 heads = 144 attention heads, token example probabilities like “floor” at 7.64%, Swedish prompts cost ~83 tokens while Finnish ~103, and temperature/top‑P and beam search alter creativity and safety.
AI-written summary · about
chapters (14)
- 0:00Welcome
- 3:57Next token prediction
- 7:28Prompt engineering and RAG
- 11:23GPT-2 to modern models
- 17:39Tokenization role
- 23:39English as programming language
- 29:22Embeddings and vectorization
- 34:40RNNs to Transformers
- 38:13Debugging GPT-2
- 45:37Next token demo
- 47:18Controlling randomness
- 53:03Top P sampling
- 56:34Beam search
- 59:24Key takeaways and closing remarks
related talks
47mInside GPT – Large Language Models Demystified
01h 01mInside GPT – Large Language Models Demystified
43mBeyond the Hype: A Realistic Look at Large Language Models
30mLarge Language Models and Prompt Engineering for Developers
41mInside GPT – The Maths Behind the Magic
25mEmbeddings, Transformers, RLHF: Three key ideas to understand ChatGPT
51mLarge Language Models for Real-World Applications - A Gentle Intro
28mGPT or how I learned to stop worrying and love the generative AI