The Built-in Browser-based Babelfish
summary
A browser-native speech-to-speech translator is built using the Web Speech API, Chrome’s experimental Prompt API with Gemini Nano, MediaRecorder/Web Audio to capture audio, and speech synthesis for output; the translator and language-detector models are tiny (translator ~80 MB) and run locally so the demo becomes offline-capable and translates English↔Spanish in real time. It works in Chrome Canary but requires heavy system thresholds (about 22 GB free, >4 GB VRAM) and currently supports only a few languages.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 3:12Building translation app
- 8:51Speech APIs limitations
- 13:07AI models in browser
- 19:34Live demo Babelfish
- 24:14Use cases for AI
- 27:01Browser AI limits
- 29:10Compatibility standards
- 31:18Developer considerations
- 32:45WebAssembly alternatives
- 35:16Model size handling
- 36:40Q&A
related talks
09mWeb apps that talk
19m"Bringing Voice First Experiences to the Web"
21mI Didn't Know The Browser Could Do That!
39mI didn’t know the Browser could do that
16mSpeech Recognition on the Web
24mAI right in the browser with Chrome's built-in AI APIs
27mWeb AI: Magic or Myth
34m2023's most interesting browser APIs