Strike a Pose - Gesture recognition in JavaScript w/ ML & Arduino
summary
Builds a gesture recognizer that actually maps live accelerometer+gyroscope traces to actions using Johnny-five on Arduino and TensorFlow JS for modeling. Data collection uses a button and an NPU1650 giving six channels, records ~50 lines per gesture (≈300 inputs), converts to tensors, shuffles, uses an 80/20 train/test split, and saves the model. Demos alternate: Bluetooth controllers often fail on stage but a phone (generic sensor API) prediction works reliably; trained model successfully classifies punches and wand gestures.
AI-written summary · about
chapters (4)
related talks
22mStrike a Pose: Gesture Recognition in JS with Machine Learning
30mSimple JStures can go a long way
42mExploring Alternative Interactions in JavaScript
49mTurn your smartphone into Nintendo Wii-like controller
05mIncredible things your community is creating with TensorFlow.js
30mLook mum, no hands ! Mind control in JavaScript
33mLook mum, no hands!
10mIMUs FTW!! Building IMU-based gesture recognition!