SymmetriSense: Enabling Near-Surface Interactivity on Glossy Surfaces using a Single Commodity ...
summary
SymmetriSense localizes a fingertip in 3D above glossy surfaces using only a single commodity smartphone (Galaxy S5) and mirror reflection geometry, not extra sensors. Implementation runs real-time at 30 fps using low-res vision (8.7 ms/frame, 25% CPU), with 18 participants on a 60-inch TV yielding ~10 mm average XY error and under 2 mm Z error; 640×480 slows to 50 ms/frame. Works on mirrors, countertops and TVs and is robust to lighting except total darkness uses reflection symmetry demonstrated on everyday glossy surfaces.
AI-written summary · about
chapters (8)
- 0:00Introduction to SymmetriSense
- 2:55Limitations of existing methods
- 4:44Using reflections for fingertip tracking
- 8:00Fingertip extraction technique
- 9:50Real-time performance and accuracy
- 13:25Z-height accuracy and robustness
- 15:11Summary and future work
- 15:34Q&A session
related talks
17mAccurate and Low-Latency Sensing of Touch Contact on Any Surface with Finger-Worn IMU Sensor
12mEclipseTouch: Touch Segmentation on Ad Hoc Surfaces using Worn Infrared Shadow Casting
14mDeepFisheye: Near-Surface Multi-Finger Tracking Technology Using Fisheye Camera
18mCyclopsRing: Enabling Whole-Hand and Context-Aware Interactions Through a Fisheye Ring
06mShadowSense: Detecting Human Touch in a Social Robot Using Shadow Image Classification
05mReady, Steady, Touch! — Sensing Physical Contact with a Finger-Mounted IMU
08mCapturing Tactile Properties of Real Surfaces for Haptic Reproduction
20mToward Realistic Hands Gesture Interface: Keeping it Simple for Developers and Machines