Open Source Voice AI: From ChatGPT's Infrastructure to Your App
summary
LiveKit shows how ChatGPT's Advanced Voice Mode runs on an open source real‑time stack, with a working hotel‑booking demo that fires async tool calls so the agent keeps talking while lookups run. A tiny turn detection model halves interruptions (9.9% → 4.5%), and measured time to first audio targets a 500–600 ms sweet spot (1.5 s is the edge). An open benchmark runs simulated callers against full pipelines to pick the best ST/LLM/TTS combo for your use case.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 3:22Why voice agents fail
- 6:33Leaderboard trap in model selection
- 9:44Perceived latency vs metrics
- 12:37Humanlike turn-taking
- 15:29Latency budget
- 17:56LiveKit voice agent benchmark
- 19:40Checklist for voice agents
- 20:42Q&A
related talks
24mBuilding a Viral Open Source AI Chatbot: Journey from Concept to Reality
33mKeynote: Fine-tuning our way towards openness in AI
37mEmpowering Developers with local AI
45mMore voice, less choice: The rise of voice interfaces and the decline of open source voice
43mAnatomy of an open source voice assistant
28mPanel: The importance of open models
58mAI in Reality Fireside Chat: Enterprise AI & Open‑Source Innovation
19mUsing OSS models to build AI apps with millions of users