A hacker's guide to open source LLMs
summary
Explains the three-stage LM pipeline: pretraining, instruction-tuning, and RLHF, and claims GPT-4 is the best model as of Sept 2023; demonstrates concrete tools and timings: GPTQ quantized 13B runs ~270ms locally, 16-bit BFloat16 runs ~390ms, 8-bit works with modest memory. Shows building a local code-interpreter via OpenAI function calling, and fine-tunes a model to turn English into SQL using Hugging Face + Axolotl + QLoRA, completing training in about an hour and producing correct queries on examples.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 1:10What is a language model
- 4:42How models learn from text
- 9:22Fine-tuning models for tasks
- 13:14Using GPT-4 effectively
- 25:04Code interpreters aren't magic
- 28:58Building a custom interpreter
- 35:00Running LLMs locally
- 46:19Improving answers with retrieval
- 55:12Fine-tuning for specific tasks
- 1:00:59Closing thoughts
related talks
27mProductionizing Open Source LLMs
35mOpen-source AI: why it matters and how to get started
24mHarnessing the Power of LLMs in Advanced Analytics
43mBeyond the Hype: A Realistic Look at Large Language Models
43mState of GPT
29mRunning Large Language Models on Laptops...
30mLearning to wield large language models
01h 49mAI engineering with open access LLMs that lie, curse, and steal