Low Level Technicals of LLMs
summary
Hands-on workshop shows exactly what was fixed and how: debugged Gemma and Phi‑3 implementations (including a rope float32 fix that causes the largest performance jump), found tokenization and untrained‑token bugs in Llama‑3, and diagnosed learning‑rate and precision pitfalls. Demonstrates Unsloth finetuning tooling that makes training 2x faster and uses 70% less VRAM, plus Colab notebooks that automate chat‑template and modelfile creation, chunked cross‑entropy for large vocabularies, and quantization/export workflows (works end‑to‑end in practice).
AI-written summary · about
chapters (17)
- 0:00Workshop introduction
- 1:25Talk agenda overview
- 3:56Bug analysis in language models
- 10:44Unsloth introduction
- 21:21Transformer architecture deep dive
- 30:28Tokenization in language models
- 44:03Training fundamentals
- 1:00:01Training complexity O(n²)
- 1:01:25Attention mechanism explained
- 1:21:01Rope embeddings and context
- 1:34:14Llama architecture investigation
- 1:57:40Finding and fixing Gemma bugs
- 2:18:43Floating point precision
- 2:33:33Gemma fine-tuning results
- 2:40:35Tokenization fixes
- 2:43:06Fine-tuning workflow
- 2:50:09Q&A
related talks
01h 49mAI engineering with open access LLMs that lie, curse, and steal
29mLocal LLM fine-tuning: a practical example
40mTechniques to avoid common LLM pitfalls
05mBuild your own LLM on Cloud
09mUsing and fine-tuning Gemma 3
46mThings you never dared to ask about LLMs
01h 21mTLMs: Tiny LLMs and Agents on Edge Devices with LiteRT-LM
09mTailoring Llama for Your Needs: Fine-Tuning & Evaluation