LLMs: Beyond the Hype - A Practical Journey to Scale
summary
Describes concrete cost-quality tradeoffs: use smaller models or LLM cascading and request hedging to route simple queries to cheap models and reserve expensive ones for hard tasks, add an LLM cache and monitoring with LangSmith or Seldon Core V2, and use vector stores like Pinecone, Chroma or PGVector for RAG. Reports quantization shrinking a 138 GB fp16 model to 40.7 GB (~3–4× compression), advocates on‑prem inference and fine‑tuning open‑source models with LangChain and LlamaIndex.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 1:10LLMs beyond hype and scaling
- 2:37LLM application anatomy
- 6:32Prompt engineering and vector DBs
- 9:30LLM agents and frameworks
- 11:56Monitoring and responsible AI
- 16:59Seven patterns for LLM systems
- 22:03Enterprise LLM challenges
- 23:30Scaling LLMs with PSA LMA RUF LMC
- 27:45LLM compression techniques
- 29:11Q&A
related talks
24mLLM Inference Optimization From Simple To Sophisticated
11mWhen Less Is More: A Technical Overview of LLMs and the Strength of Smaller Models
24mCost and Performance Optimization of LLM Inferencing.
33mThe Right Model for the Right Job: Cutting LLM Costs Without Cutting Quality
25mvLLM: Easy, Fast, and Cheap LLM Serving for Everyone
28mNavigating LLM deployment: Tips, tricks, and techniques
01h 49mAI engineering with open access LLMs that lie, curse, and steal
15mvLLM: Easy, Fast, and Cheap LLM Serving for Everyone