Wearable Live Captions

on YouTube ↗38m

" summary

Built a flexible 28×8 NeoPixel LED array sewn into a mask that scrolls live captions in real time using Azure Cognitive Services for speech‑to‑text and an Ably MQTT broker to push compact binary messages to a tiny Wi‑Fi microcontroller. Audio and transcription run on a phone to offload work, the device embeds a 4×8 pixel font and decodes STX/ETX framed byte streams, and scrolling is implemented by shifting an X offset so it uses minimal memory and works reliably while walking.

AI-written summary · about

" chapters (8)

1. 0:00Start
2. 11:42USING ACS SPEECH SDK
3. 15:53AN EXAMPLE COMMAND
4. 20:19THE DISPLAY DRIVER
5. 22:39CREATING A PIXEL 'FONT'
6. 24:58HARDWARE DIFFERENCES
7. 30:03WRITING TEXT TO THE DISPLAY
8. 32:24SCROLLING THE TEXT

" related talks

• 43mWearable Live CaptionsDavid Whitney · Build Stuff 2022
• 37mWearable Live CaptionsJo Franchetti · NDC Oslo 2022
• 14mWearable Subtitles: Augmenting Spoken Communication with Lightweight Eyewear for All-day CaptioningThad E. Starner, Bonnie Chinh, Alex Olwal, Kevin Balke, Dmitrii Votintcev, Paula Conn, Benoit Corda · UIST 2020
• 45mBe My Eyes - Leveraging Generative AI to help the blind and visually impaired peopleDanushka · Devoxx Belgium 2024
• 49mMaking IoT: An Intro for Web DevsKristina Durivage · NDC Oslo 2019
• 28mCode you can hold: Making your first IoT wearableCharlyn Gonda · TechChill 2019
• 05mLive Captioning with White Coat CaptioningNorma Miller, Andrew Howell · AngularConnect 2017
• 55mMaking Every Game and App More Accessible with .NET and AIAlex Dunn · NDC Oslo 2022