Strike a Pose: Gesture Recognition in JS with Machine Learning
summary
Builds a realtime gesture recognizer using Arduino + MPU6050 motion data sent over Node.js (johnny-five) into files, converts those text recordings into tensors, shuffles and splits 80/20, trains a sequential TensorFlow.js model via fit, saves the model and predicts live gestures as numeric indices. Prototypes include a Street Fighter body‑move game and wandlike “spells” demo, records roughly hundreds of samples per gesture, and notes visible latency and occasional demo crashes.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 1:30Gathering Data
- 4:03Code
- 5:38Data
- 11:27Model
- 12:39Prediction
- 14:35Demo
- 18:29What Else
- 19:10Generic Sensor API
- 19:59Recap
- 20:44Conclusion
related talks
28mStrike a Pose - Gesture recognition in JavaScript w/ ML & Arduino
19mHand gesture-controlled robot through the Internet
30mSimple JStures can go a long way
05mIncredible things your community is creating with TensorFlow.js
42mExploring Alternative Interactions in JavaScript
28mBuilding Motion Controlled Websites in JavaScript
30mLook mum, no hands ! Mind control in JavaScript
13mml5.js - Friendly machine learning for the web