Discussion Session
Talkat Special Year on Large Language Models and Transformers, Part 1 Boot Camp · 2024-09-04..09-06·
summary
A wide-ranging discussion maps specific research bets and failures: controlled data and open‑source training to test combinatorial creativity and diffusion claims; studies that match ImageNet weight statistics (Zico Kolter group) suggesting simpler pretraining explanations; benchmark failures like the ARC Prize that force search or heavy inference; the need to study latent state and external memory; moving compute from inference back to training; algorithms glued to LLMs (AlphaGo style MCTS + nets) and the promise of small models with smarter pre/post‑training and better evaluations.
AI-written summary · about
chapters (8)
- 0:00Session introduction
- 5:05Research directions
- 12:41Small models and algorithms
- 21:24Prediction vs generation
- 28:44LLM capabilities and gaps
- 37:01Learning through analogies
- 47:33Tool use and future models
- 56:47Final reflections on training
related talks
36mPanel Discussion
54mKeynote: Can you trust your (large language) model?
40mPannel Discussion | From Hype To Value: Real World Applications Of Generative AI And LLMs
35mFireside Chat
44mKeynote: Tracking the Pulse of Generative AI
43mDiscussion
37mRAG, Tooling, MCP, Agents ... and What is Missing?
33mLive Q&A Session 2