Panel Discussion
summary
Quantized training and inference win real gains here: BitsAndBytes delivers 8‑bit optimizers and QLoRA proves low‑cost fine‑tuning, while an open LM Eval Harness and the Pythia checkpoints document pretraining data and ordering for reproducibility. Teams report chaining test‑time compute (O1 style) to boost reasoning, distilling teacher outputs into much stronger 7B–12B models with claimed 2–3x efficiency gains. Multimodal demos include an “advanced voice mode” with emergent conversational quirks and SweetBench‑style grounding for fixing real GitHub bugs.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 1:03Panelists' AI journeys
- 7:02Current AI landscape trends
- 16:04Test-time compute and reasoning
- 22:45Multimodal LLMs and future
- 30:44Predictions for next 12 months
- 35:58Closing remarks
related talks
31mMaking the most of test-time compute in LLMs
42mPost-Training Small Language Models: the adventures of a practitioner
25mWeights vs. Words in Fine-Tuning & Enhancing Agentic AI Workflows
25mThe Golden Triangle of Inference Optimization: Balancing Latency, Throughput, and Quality
01h 07mDiscussion Session
02h 52mLow Level Technicals of LLMs
42mFine-tune and deploy LLMs from Hugging Face (AIM304)
32mScaling Test-time Inference Compute & Rise of Reasoning Models