Pentelligence: Combining Pen Tip Motion and Writing Sounds for Handwritten Digit Recognition
summary
Pentelligence is a USB-wired pen that logs IMU motion, a shaft microphone (Hilbert envelopes), and a binary contact sensor, feeds mean-filtered motion (150 samples/axis) and 3500-input audio vectors into separate neural nets, then resolves labels by majority/verification voting. Evaluation uses 9,408 digits from 26 people and yields 78.4% accuracy on 10 test users; retraining per writer on 2,120 samples gives 100% single-digit precision on 2120 samples and 98.3% overall accuracy. Audio can be disabled when ambient noise is high.
AI-written summary · about
chapters (7)
- 0:00Introduction and motivation
- 2:12Pentelligence pen prototype
- 5:16Audio data processing approach
- 7:01User study and evaluation
- 9:12Combining motion and audio data
- 11:59Single-writer performance and applications
- 13:34Q&A session
related talks
20mUnimanual Pen+Touch Input Using Variations of Precision Grip Postures
41mCreating Intelligent Music Instruments with Machine Learning
20mAccelerometers as an Instrument
20mNew User Interface - Part 1
10mIMUs FTW!! Building IMU-based gesture recognition!
28mStrike a Pose - Gesture recognition in JavaScript w/ ML & Arduino
33mBuilding "learn to touch type" glove with ...
22mFingerIO: Using Active Sonar for Fine-Grained Finger Tracking