Machine learning for Accessibility

on YouTube ↗12m

" summary

Shows concrete systems and numbers: IconNet runs locally to label ~30 icons at launch then +40 more, filling missing accessibility labels; ML Kit powers Lookout with careful precision/recall tuning for currency (high precision) versus room-object recognition (balanced recall); Live Transcribe uses Cloud Speech‑to‑Text plus YamNet for 600+ sound classes and adds sound notifications with haptic smartwatch alerts. Includes open‑source apps and a gesture toolkit, plus implementation details and tradeoffs rather than high‑level promises.

AI-written summary · about

" chapters (4)

1. 0:00Start
2. 4:19Lookout
3. 6:26Use Cases
4. 11:07Bonus: Project Shuwa Sign Language Detection and Deaf Culture Awareness

" related talks

• 31mPower-Up Your App With Apple’s Machine Learning FrameworksKim Foong Chow · iOS Conf SG 2017
• 39mCustomize Your App with MLKitVictoria Gonda · Øredev 2019
• 47mBuilding smarter apps with Machine Learning, from magic to realityLaurent Picard · GDG DevFest Poland 2018 in Wrocław
• 24mML Kit: Machine learning for mobile developersSachin Kotwani, Isabella Chen · Firebase Summit 2018
• 42mSmartly Smartest Apps with Cross-Platform Machine Learning (Using Flutter and ML Kit)A. CatalaÌn · Android Summit 2019
• 28mMachine Learning applications in accessibilitySheri Byrne-Haber · Data Con LA 2020
• 46mArtificial intelligence to the rescue of accessibilityGuillaume Laforge, Aurélie Vache · Devoxx France 2021
• 23mDesigning for AccessibilityElise Roy, Michael Brenner, Irene Alvarado · Google I/O 2019