What Lies Beneath the API
summary
Intercom reports one tenth the cost versus frontier APIs and Pinterest claims orders of magnitude gains, and Modal argues that maturing products move toward fine‑tuning. Concrete signals: paying more for the API than customers pay, evals plateauing, and latency requirements no shared endpoint meets. Supervised fine‑tuning fits in 300 lines of Python, RL rollouts scale to 50,000 sandboxes, and deployment uses vLLM SG Lang Triton or custom inference with Modal for autoscaling.
AI-written summary · about
chapters (6)
- 0:00AI app development patterns
- 1:51Frontier APIs limitations
- 3:19Traditional fine-tuning challenges
- 5:50Time for fine-tuning
- 8:14Serverless training tools
- 11:42When to train model
related talks
14mAPI-Driven Architectures Deliver 60% Faster Time-to-Market
23mMachine Learning powered API governance
24mAPI security, mitigate API abuses using machine learning
23mLive Q&A
27mPlatforms for valuable AI Products- Iteration, iteration, iteration
15mHow Modern Architecture Slashed Cost & x2 Deployment Speed
06h 28mDay 5
26mMLOps for the rest of us- A poor man's guide to putting models in production