State of GPT
summary
Describes the concrete GPT pipeline: large-scale pretraining that uses thousands of GPUs for months (example: LLaMA 65B trained on 1.4 trillion tokens using ~2,000 GPUs, ~21 days), followed by supervised finetuning on tens of thousands of prompt–response pairs and an RLHF reward model trained from human rankings. Shows RLHF improving human preference but reducing entropy, recommends retrieval-augmented prompts, calculator/tool calls, Python “glue” for Tree-of-Thought/ReAct search, and LoRA-style parameter-efficient fine-tuning.
AI-written summary · about
chapters (12)
- 0:00Intro
- 0:58GPT Assistant training pipeline
- 2:04Data collection
- 3:232 example models
- 8:13Base models learn powerful, general representations
- 13:24RM Dataset
- 14:05RM Training
- 18:47Mode collapse
- 25:50Ensemble multiple attempts
- 26:47Ask for reflection
- 32:13Tool use / Plugins
- 37:15Default recommendations
related talks
01h 02mA hacker's guide to open source LLMs
01h 38mTraining Large Language Models: Practices and Research Questions
19mSystem for Scaling Large Language Models
45mHow to use GPT in your projects: A Case Study
01h 00mInside GPT – Large Language Models Demystified
39mCollect Data to Fine-Tune Your Large Language Model
25mEmbeddings, Transformers, RLHF: Three key ideas to understand ChatGPT
43mBeyond the Hype: A Realistic Look at Large Language Models