"Bringing Voice First Experiences to the Web"
summary
Demonstrates on‑device speech recognition via the browser SpeechRecognition API with phrase events and confidence scores, uses the Chrome Canary prompt API to run Gemini Nano locally for cost‑effective LLM sessions, and plays responses with the speech synthesis API through a speech synthesis utterance that can pick voices and adjust pitch and rate. Notes real problems with interruptions and noisy environments, reports a working websocket pipeline and a deployable client via the Hume React SDK that streams audio, returns expressions, and serves text→speech and speech→speech.
AI-written summary · about
chapters (10)
- 0:00Voice-first web intro
- 1:06Why voice matters
- 3:13UX flow for voice
- 4:16Speech recognition API
- 6:49LLMs with on-device models
- 9:42LLMs triggering side effects
- 11:31Speech synthesis for audio
- 14:44Natural conversation challenges
- 16:33Hume’s React SDK
- 19:03Closing & Q&A
related talks
39mThe Built-in Browser-based Babelfish
38mIntegrating AI and Browser API Voice Recognition on Web App
16mSpeech Recognition on the Web
09mWeb apps that talk
38mConversational UIs: Talking to Siri, Alexa, and Your Web Browser
21mBuilding a Voice-Enabled AI Assistant With Javascript
29mFinding Your Voice: Building Screenless Interfaces with Node.js
48mConversational ULS: Talking to Siri, Alexa, and Your Web Browser