Latent Space Paper Club: AIEWF Special Edition (Test of Time, DeepSeek R1/V3)
summary
DeepSeek R1 (May 28 update) claims large reasoning gains by extending test‑time compute and pure RL: AIM score rises 70→87.5 and average reasoning traces double from ~12,000 to ~25,000 tokens. Training uses a four‑stage pipeline: cold‑start SFT → gRPO RL → rejection sampling → final RL polish. Resulting builds add native JSON and function‑calling, reduce hallucinations, and a distillation run produces an 8B model matching a 235B reasoning model on many benchmarks. Paper Club also launches a six‑month test‑of‑time curriculum covering core papers.
AI-written summary · about
chapters (13)
- 0:00Paper Club year in review
- 1:23Test of Time Paper Club launch
- 7:45DeepSeek R1 May 28 update
- 15:32Original DeepSeek paper recap
- 22:32Inference time scaling explained
- 28:54DeepSeek V3 model details
- 31:29DeepSeek R10 reasoning model
- 38:49From R10 to DeepSeek R1
- 44:11Distillation models performance
- 47:48Future work and open recreations
- 48:08Second Paper Club announcement
- 49:54Recap of DeepSeek updates
- 52:23Aha moments and inference scaling
related talks
25mFrontier AI Models: A Review of Past Nine Months
15mOlmo-Thinking: Training a Fully Open Reasoning Model
24mThe Rise of Reasoning Models
29mAligning DeepSeek R1
01h 05mAdaptive Generalization: The Role of Dynamic Evaluation in Low-Resource Settings
39mUnlocking reasoning in Large language models
01h 05mCan you trust your (large language) model?
20mEverything I Learned Training Frontier Small Models