SignGlass: First-Person View Comprehensive and Generalizable ASL Translation Using Wearable Glass
summary
A wearable glasses prototype with three onboard micro‑cameras captures both hands and face and runs on-device streaming over Wi‑Fi to produce real‑time ASL→English translation; the pipeline uses a jitter‑aware spatio‑temporal attention hand branch, a dual‑camera temporally‑aware facial module, and cascaded data augmentation for generalization. A controlled dataset from 14 Deaf participants covering ~200 signs yields hand‑only error >30% dropping to ~18% with facial cues and translation quality of BLEU >80%; users rate comfort near 9/10.
AI-written summary · about
chapters (7)
- 0:00Introduction and motivation
- 1:10SignGlass system overview
- 3:16Prototype design and data collection
- 4:42System pipeline and translation process
- 6:28Validation and user study results
- 7:11Social acceptance and future work
- 7:55Q&A session
related talks
06mTeaching American Sign Language in Mixed Reality
01mEnabling Real-time Sign Language Translation on Mobile Platforms with On-board Depth Cameras
35mHow deep learning can help interpret American Sign Language
09mSmart Glasses for the blind
45mBe My Eyes - Leveraging Generative AI to help the blind and visually impaired people
09mBeyond Captions A Practical Pipeline for Live AI-Generated Sign Language Avatars
48mWearAbility: the connection between accessibility and wearable computers
14mTransform intent into action through your eyes