Laptop-sized ML for Text, with Open Source
summary
Demonstrates running modern open models locally: LLaMA on a laptop (released in 13–122 GB sizes) and several community forks and tools (OpenLLaMA, MPT, llama.cpp, Hugging Face). Shows embeddings vs TF‑IDF, uses mxnet examples, and claims fine‑tuning yields big behavior changes — Stanford’s Alpaca via LLaMA fine‑tune costs about $100, Databricks crowdsourced Dolly 15K Q&A, and Guanaco achieves similar accuracy with a 5 GB quantized model. Covers failures: bias, hallucinations, token limits, prompt injection and licensing/copyright risks.
AI-written summary · about
chapters (23)
- 0:00Introduction and talk overview
- 1:45What is a large language model
- 3:53Old models still relevant with LLMs
- 5:43Demo: Running LLaMA on a laptop
- 6:05Key ML and IR terms explained
- 7:30Older models and their capabilities
- 11:02LLM revolution and timeline
- 13:33Facebook's LLaMA model details
- 15:17Open alternatives to LLaMA
- 17:42Fine-tuning models with Alpaca
- 19:07Cost and accessibility of fine-tuning
- 21:13Commercial open models from MosaicML
- 23:20OpenLLaMA and Guanaco models
- 25:46Optimizing models and vector search
- 26:07Eight things to know about LLMs
- 28:41Using LLMs for embeddings and relationships
- 30:26Constraining LLM output with JSON
- 32:11Token limits and context windows
- 34:19How context affects LLM responses
- 36:05Problems and failure modes with LLMs
- 39:16Running LLMs on your laptop
- 40:22Resources and next steps
- 41:49Q&A and closing remarks
related talks
20mSelf-hosting AI LLMs - a beginners guide
01h 35mSave your API Keys for someone else
29mRunning Large Language Models on Laptops...
31mDeploying Large Language Models with Ease
38mMeta's Roadmap for Full Stack AI
01h 19mBuilding an LLM on your Laptop to...
18mSmolLM: The Rise of Smol Models
38mUnraveling Multimodality with Large Language Models