Why ChatGPT Keeps Interrupting You
summary
Voice agents use a simple VAD plus a 0.5s silence rule today, which causes most interruptions. LiveKit’s transformer-based text-based semantic model feeds the last four turns (agent, user, agent, user) to predict an end-of-utterance token and extends silence when prediction says “not done,” and AssemblyAI’s streaming STT emits transcript plus finish-probability. Experimental full duplex models (Moshi, Sync LLM) continuously listen and emit silence while forecasting ~5 tokens/200ms. Demos show big improvement but no approach fully solves backchannels or production control needs yet.
AI-written summary · about
chapters (13)
- 0:00Intro
- 1:07What is turntaking
- 2:21How voice AI agents work
- 3:19Voice activity detection
- 4:09How human turntaking works
- 5:26Three stages of turntaking
- 7:11The X axis
- 8:49Three types of models
- 9:27Textbased semantic models
- 10:42Demo
- 12:57Other approaches
- 18:11Future predictions
- 19:00QA
related talks
33mDesigning Voice Agents for Real Conversations
22mVoice Agents That Think While They Talk
13mVoice In, Visuals Out: The Agony and the Ecstasy
27mWhat happens when AI starts grocking streaming audio directly
01h 01mHow ChatGPT Works
02h 00mChatGPT & Prompt Engineering
35mChatGPT is lying, how can we fix it?
55mEveryday AI: Practical Applications Beyond the Chatbot