Parallel Processors: Past & Future Connections Between LLMs and OS Kernels.
summary
Argues that Transformers are a new kind of computer — a differentiable natural language processor whose core is content‑addressable vector memory rather than addressable bits — and maps OS ideas to LLMs: paged KV cache (VM inference) cuts memory use and enables beam/parallel sampling, speculative decoding gives 2–3× throughput by drafting tokens in parallel, demos like HustleGPT show limits, and gaps remain in isolation, interrupts, open standards and prompt leakage that break multiuser, secure deployments.
AI-written summary · about
chapters (6)
- 0:00Introduction to talk
- 1:48What are LLMs as new computers
- 14:28Historical demos and LLMs
- 19:02Efficiency gains from OS concepts
- 27:29LLM operating systems vision
- 32:05Recap and call to action
related talks
30mLLM Inference Arithmetics: the Theory behind Model Serving
45mLarge language models are not a paradigm shift
49mPushing LLMs over the Edge: Exploring the Limits of the Possible
15mvLLM: Easy, Fast, and Cheap LLM Serving for Everyone
25mvLLM: Easy, Fast, and Cheap LLM Serving for Everyone
49mIntelligent Interfaces and operating systems
37mExpanding the Capabilities of LLMs Through Community Collaboration
44mThe LLM Revolution: From Inception to the Future of Thinking Machines