Live voices with LiveView
summary
A working local voice assistant pipeline is built that runs a small LLM (135M params) plus Whisper for transcription, a lightweight VAD, and a Kokoro-like TTS, all orchestrated with Membrane and exposed via LiveView; models run on a MacBook Air with ~11 GB free RAM and use 16 kHz / 24 kHz sampling. The stack uses Bumblebee/Nx and ONNX runtimes, keeps heavy models resident to avoid startup costs, and the demo produces audible responses (quirky “Emily” florist persona) with reasonable latency.
AI-written summary · about
chapters (19)
- 0:00Welcome
- 1:27Live demo: Emily
- 2:51ML models overview
- 5:00ML local challenges
- 7:26Small language model
- 9:12Voice activity detection
- 10:59NX and ONNX models
- 13:49Nyquist and Shannon theorems
- 16:20Audio pipeline building
- 17:45Membrane framework
- 19:09Media processing challenges
- 21:16Audio pipeline demos evolution
- 25:18Membrane architecture
- 27:47Membrane plugins
- 30:59LiveView UI integration
- 32:04Scaling voice assistant
- 35:45Elixir orchestration
- 36:07Live coding language model
- 38:38Q&A
related talks
27mBuilding a Low-Latency Voice Assistant leveraging Elixir and Membrane
30mLook ma, no hands! Multimodal AI-Agents in the browser
01h 04m“Hello, AI?!” – Real-Time Voice Interactions with LLMs
49mHello JARVIS - Building Voice Interfaces for Your LLMS
33mDesigning Voice Agents for Real Conversations
21mMultimodal Open Source at Kyutai, From Online Demos to On-Device
29mFinding Your Voice: Building Screenless Interfaces with Node.js
22mBoring but Effective: WebAI That Ships