How We Built a Voice Agent with Temporal
summary
Built a live phone agent that reuses a text-based Temporal agent, adds Deepgram STT and Cartesia/ElevenLabs TTS, and speeds LLMs with Grok and Cerebras plus caching and small-model narrowing. To hit voice latency the team traded resilience for speed, running three parallel LLM requests per candidate and using detached child workflows (wrapper workflows started from side effects) with a fast-exec library. Temporal adds ~30 ms per nested start and overall V1 overhead ranges ~1–200 ms; the system works but leaves 100–200 ms on the table.
AI-written summary · about
chapters (9)
- 0:00Introduction to voice agent
- 2:13Building text-based AI agent
- 7:08Exploring voice agent options
- 12:02Incorrect assumptions about latency
- 17:18Temporal workflow latency issues
- 21:49Solving the workflow bottleneck
- 28:03Optimizing for speed and debugging
- 29:46Final verdict on Temporal for voice
- 30:08Q&A session
related talks
17mBuilding Effective Voice Agents
33mDesigning Voice Agents for Real Conversations
25mEngineering voice agents: Latency, quality, and scale
20m10 Things I Learned Building Voice Agents in 2025
17mShipping an Enterprise Voice AI Agent in 100 Days
27mBuilding a Low-Latency Voice Assistant leveraging Elixir and Membrane
13mVoice In, Visuals Out: The Agony and the Ecstasy
01h 26mBuilding voice agents with OpenAI