Building Reliable LLM Systems
summary
Models stall not from performance but from human queues: the talk shows decision latency rising from seconds to days, recommends a three‑tier gate (pre‑authorized, async review with a 4 hours SLO, explicit approval for irreversible changes), and makes policy changes executable with policy as code so prompt checks, data classification, and thresholds run in milliseconds with audit logs tagging model and policy versions. It treats decision latency and cognitive load as SLOs and gives four actionable steps: map loops, assign owners, automate one rule, and publish a golden path.
AI-written summary · about
chapters (13)
- 0:00The Hidden AI Stall
- 0:49Bottleneck Shift
- 2:06Decision Latency Load
- 3:23Governance Queues
- 4:59Burnout Complexity
- 6:44SRE Lens Expanded
- 8:15Explicit Boundaries
- 9:55Ownership Clarity
- 11:24Policy as Code
- 13:13Golden Path Defaults
- 16:12Sustainable Velocity
- 17:20Next Week Actions
- 18:51Closing Thanks
related talks
56mHerding LLMs: Predictable Outcomes from Unpredictable Agents
32mThe Sorcerer’s Apprentice: The craft of building reliable solutions with LLMs
20mBuilding Deterministic Infrastructure for Non Deterministic AI Agents
47mA Step-by-Step Guide to Securing Large Language Models (LLMs)
06mObservability for Large Language Models
38mBehind the Firewall: Designing Secure LLM Pipelines for Regulated Data
28mNavigating LLM deployment: Tips, tricks, and techniques
27mThe Lean LLM development playbook