How agents broke app-level infrastructure
summary
LLMs force app latencies into the multi-second range so existing serverless expectations break: p1 latency of a couple of seconds and many serverless providers time out after 5 minutes. The solution described is a separation of API and compute, durable execution with heartbeats, retries, traces and Redis streams for resumability, plus a serverless compute layer on ECS and tooling to stream intermediate outputs. An open source library implements these pieces and is reported to work for multi-minute agentic workflows.
AI-written summary · about
chapters (7)
- 0:00The shift in AI engineering
- 1:21Infrastructure limitations
- 2:50The serverless gap
- 4:03Real-world agent UX
- 7:17A component-based framework
- 9:26Architecture for reliability
- 10:55Lessons and best practices
related talks
44mApplying principles of chaos engineering to serverless
40mThe State of Serverless Computing
43mApplying the Principles of Chaos to Serverless
15mServerless & Agentic AI: Better Together
39mBuilding Resilient Serverless Applications
35mEfficiency of Serverless Architectures with AWS Solutions
43mBest Practices for Serverless Developers
01h 00mMicroservices and Serverless