Building voice agents with OpenAI
summary
Claims that voice agents are an API to the real world and that OpenAI now ships an OpenAI Agents SDK for TypeScript with native voice support and guardrails; demo builds a browser agent using WebRTC real-time session, Next.js and tools (weather, refund) that delegates heavy reasoning to an O4‑mini backend, handles interruptions and human approvals, and stores transcripts in memory while sending traces to the traces dashboard for replay. The live demo mostly works (network echo causes a few miscounts—agent stops counting at six when interrupted) and shows lower-latency speech-to-speech, chained text fallbacks, and practical tool-calling.
AI-written summary · about
chapters (7)
- 0:00Timestamps
- 0:16Introduction to voice agents.
- 1:28Overview of the OpenAI Agents SDK for TypeScript.
- 3:27The case for why voice agents are important.
- 4:21A look at different architectures for voice agents.
- 1:00:16Best practices for building voice agents.
- 1:17:31A hands-on guide to building a voice agent.
related talks
23mBuilding with OpenAI: What's Ahead
21mBuilding a Voice-Enabled AI Assistant With Javascript
20m10 Things I Learned Building Voice Agents in 2025
33mDesigning Voice Agents for Real Conversations
30mLook ma, no hands! Multimodal AI-Agents in the browser
29mClient side Web AI Agents Building smarter user experiences for a future age
56mAI Agents That Don’t Suck: How to Build Ones That Actually Drive Business Value
40mAI Agents in Practice