Write, Attend and Spell: Streaming End-to-end Free-style Handwriting Recognition Using Smartwatches
summary
A smartwatch system captures accelerometer, gyroscope and gravity signals and runs on an Android prototype to map motion to whole words using a multimodal CNN for feature fusion and a stacked dilated convolutional encoder‑decoder with a CTC streaming decoder for real‑time letters. Trained on data from 12 volunteers and 250 words, it reports 7.4% CER for new users, 3.7% CER on unseen words and word accuracies of 84.3% and 88.2%, and tolerates different watches and surfaces.
AI-written summary · about
chapters (8)
- 0:00Intro
- 0:23Motivation
- 1:49Data Collection
- 2:08Handwriting Recognition
- 3:03Experiment Setup
- 4:43System Performance
- 4:55User Survey
- 5:14Conclusion
related talks
21mThe Impact of Word, Multiple Word, and Sentence Input on Virtual Keyboard Decoding Performance
16mMyoSign: enabling end-to-end sign language recognition with wearables
44mBuilding Handwriting Recognition System for the NY Times Crossword
40mBuilding a Handwriting Recognition System
25mBuilding a Handwriting Recognition System for the New York Times Crossword
13mUniKey: Enabling Surface-Based Typing with Commodity Smartwatches via Cross-Modal Learning
24mSIGN TO TEXT Unleashing the Power of Google AI for real time ASL transcription
18mOpen Source Handwriting Recognition