Your realtime AI is ngmi
summary
Argues that realtime voice AI must be built from the network layer up, not from the model or app, and demonstrates using WebRTC vs WebSockets why WebRTC handles jitter, packet loss and bandwidth estimation natively. Targets sub-1s voice-to-voice latency (500 ms feels natural) and reports 10–15% real-world connection issues if you get networking wrong. Shows a Raspberry Pi peer‑to‑peer WebRTC demo talking to cloud LLMs via Pipecat using MLX, Whisper and Gemma 3 with a custom logic sampler — the demo runs live and works.
AI-written summary · about
chapters (11)
- 0:00[Voice Keynote] Your realtime AI is ngmi — Sean DuBois (OpenAI), Kwindla Kramer (Daily)
- 1:29Introduction to Voice AI and Latency
- 2:46Latency Breakdown in a Voice AI Application
- 3:27WebRTC vs. WebSockets for Real-Time Audio
- 6:41Advantages of WebRTC
- 7:49Applications of WebRTC
- 8:52Future of Voice AI and User Interfaces
- 9:59Squabbert Demo
- 12:44Flexibility of WebRTC Connections
- 13:09Community Showcase: Yashin's Project
- 15:46Call to Action and Resources
related talks
33mDesigning Voice Agents for Real Conversations
27mWebRTC: Real-Time Communications
20mBuilding the next interface - AI personas who feel as natural as interacting with a human
01h 07mBuilding and Scaling an AI Agent Swarm of low latency real time voice bots
42mBecoming Latency-Native
21mWebRTC A Quick Introduction
43mNow Hear This! Putting Real-Time Voice, Video and Text into Rails
24mState of client side AI