Hands-free Coding with Gaze Control in JavaScript
summary
Prototype uses a webcam and tensorflow.js gaze estimation plus an electron + robot.js bridge to emit keystrokes into an editor. The author packages the gaze math as a gaze-detection npm package that normalizes iris-to-face coords and treats normalized x > 0.355 as “right” to iteratively halve a phrasebook until one snippet is selected. Demo types a short React component in ~30 seconds. Works but needs good lighting, per-editor camera limits, and has dataset bias risks.
AI-written summary · about
chapters (10)
- 0:00Start
- 0:36Charlie Gerard
- 0:59Inspiration
- 7:35Gaze Detection
- 8:00Code Samples
- 14:03React Component
- 14:33Github Co-Pilot
- 18:46Learning Curve
- 22:13How Can People Get Started with Machine Learning in Javascript
- 22:48Tensorflow Js
related talks
21mHands-Free Coding With Gaze Control in JavaScript
42mExploring Alternative Interactions in JavaScript
30mLook mum, no hands ! Mind control in JavaScript
14mLook ma, no hands! - How to program without using your hands
18mExploring alternative interactions in JavaScript
28mBuilding Motion Controlled Websites in JavaScript
19mHand gesture-controlled robot through the Internet
45mMind control in JavaScript - Experimenting with alternative interactions