Running Agents on Async Task Queues While Keeping them Real-Time
summary
ZoomInfo moves agent execution off FastAPI web pods onto Temporal task queues, keeping the browser SSE UX by publishing LLM tokens to a Redis token stream with client reconnect and no session affinity. LangGraph agent code remains unchanged, so teams avoid rewrites while gaining durability, per-agent Kubernetes scaling, backpressure handling and canary routing via LaunchDarkly (10%). Result: long-running 30+ second agents scale to thousands of concurrent agents reliably.
AI-written summary · about
chapters (7)
- 0:00Introduction and speaker intro
- 0:45Problem with FastAPI server approach
- 3:54Runtime challenges and agent crashes
- 5:18Choosing Temporal for async execution
- 7:01Migration to Temporal workflow
- 8:24Benefits of agent broker and scale
- 9:49Key takeaways and summary
related talks
15mScaling AI Agents Without Breaking Reliability
23mAgentic Elixir
25mYour Attention Is the Bottleneck, Not Your Agents
14mHow agents broke app-level infrastructure
29mRunning AI Agents Inside TEEs Without Losing Your Mind
15mConquering Agent Chaos
52mDesigning Production-Ready Agentic Systems: Lessons from the Field
40mReal-TIme Context Engineering for Agents