Building Voice AI for People Who Don’t Trust AI
summary
Built a phone-first voice agent for 55+ users that avoids apps, uses a natural female 11lab voice and is named Sophie, explicitly discloses it’s AI, and stores short-term memory with Mem Zero for personalization. Production targets hit 500–700 ms end-to-end latency; phone stack adds call-blocking, voicemail and machine-answer problems solved with layered detection: carrier Twilio AMD + heuristics + an LLM classifier trained on collected examples. SMS is charged by ~60-char segments, payment links suffer low delivery, and every minute or message directly increases operating cost.
AI-written summary · about
chapters (10)
- 0:00Intro and Background
- 0:21Why Voice Calls
- 1:37Trust and Perception
- 2:43Designing Trust Signals
- 4:53Scripts and Guardrails
- 5:32Phone Infrastructure Challenges
- 6:14Latency and Call Blocking
- 7:18Voicemail Detection Stack
- 8:42SMS Agent Pitfalls
- 11:06Costs and Key Lessons
related talks
27mWhat's Next in Voice for Brand Leaders
17mServing Voice AI at Scale
22mDesigning Voice For Humans
36mEmbodied AI: A Cognitive Shift, Not a Cosmetic Change
57mDesigning Human-AI Experience
29mAndroid Gets a Voice: Building Real-Time Agents
01h 07mBuilding and Scaling an AI Agent Swarm of low latency real time voice bots
26mInterview