GesturePod: Enabling On-device Gesture-based Interaction for White Cane Users
summary
GesturePod is a clip-on module for any white cane that uses a Bosch IMU, a Cortex M0 microcontroller, BLE and a Li-Po battery to run a ProtoNN classifier on-device; the hardware weighs less than 50 g, lasts over a day and costs under $10. The system recognizes five subtle cane gestures with 92% accuracy, speeds common smartphone tasks by 3x–10x in lab tests (12 participants) and stays in use across 15 days in a 3-user field trial, with code and datasets released at ak.ms/gesturepod.
AI-written summary · about
chapters (34)
- 0:00Introduction
- 0:18Scenario
- 0:45Voice commands
- 1:45Touchpads
- 2:10The Idea
- 2:29The Challenges
- 3:05The Hardware
- 3:57The 5 Gestures
- 4:33Classification
- 4:56Rulebased Classification
- 5:17Machine Learning
- 5:33Data Collection
- 5:50Time Series Classification
- 6:08Training Data
- 6:20Negative Examples
- 6:44Classification Algorithm
- 7:19Open Source
- 7:39Android App
- 7:57Robustness
- 8:24Impact
- 9:33Results
- 10:05Log Scale
- 10:27P3 Usage
- 10:54Feedback
- 11:21While Walking
- 11:40While Walking P1
- 12:06Android App Overview
- 12:23Summary of GesturePod
- 13:13Pipeline
- 13:30Quick question
- 13:46Pointing out P3
- 14:01Demonstration
- 14:55Dream Help
- 15:17Android Talkback
related talks
18mM.Gesture: An Acceleration-Based Gesture Authoring System on Multiple Handheld and Wearable Devices
20mNew User Interface - Part 1
07mKeep the Phone in Your Pocket: Enabling Smartphone Operation with an IMU Ring for Visually ...
18mSelf-Powered Gesture Recognition with Ambient Light
21mPosePIE: Replace Your Keyboard and Mouse With AI-Driven Gesture Control
06mDesigning and Evaluating Hand-to-Hand Gestures with Dual Commodity Wrist-Worn Devices
20mTomo: Wearable, Low-Cost Electrical Impedance Tomography for Hand Gesture Recognition
21mEnabling People with Visual Impairments to Navigate Virtual Reality with a Haptic and Auditory ...