Stop Paying for Frontier Models
summary
Explains how to replace expensive “frontier” LLM calls with smaller local models where appropriate by using concrete evaluation methods: capability evaluations, golden datasets, and LLM-as-judge. Presents three concrete cases — a local agentic harness replacing a remote model, an in-browser moderation classifier validated with production-trace evaluations, and a generative summarization feature where the rubric dominated model choice — and offers a framework and repeatable process for building evals from your own production traces to decide and measure when to use large off-prem models vs small local ones.
AI-written summary · about
related talks
31mFrontier results, on device
30mLLMs: Beyond the Hype - A Practical Journey to Scale
25mEngineering Better Evals: Scalable LLM Evaluation Pipelines That Work
33mThe Right Model for the Right Job: Cutting LLM Costs Without Cutting Quality
30mLearning to wield large language models
20mEverything I Learned Training Frontier Small Models
29mLocal LLM fine-tuning: a practical example
25mSelecting the Best LLMs with Model Cockpit