Hand gesture-controlled robot through the Internet
summary
Web app runs hand-pose detection in the browser with MediaPipe landmarks (21 points × 3 coordinates) and a custom TensorFlow.js classifier that outputs six commands from those keypoints, trained and saved to localStorage or downloaded. Front end uses React + TypeScript, getUserMedia video, and Socket.IO to send commands to a Raspberry Pi running Node.js. Demo runs around 60 fps and the pipeline performs real-time predict calls with TensorFlow.js on-device.
AI-written summary · about
chapters (9)
- 0:00Introduction and project overview
- 1:46Web application goals and setup
- 3:15What is Web AI?
- 4:43Hand gesture recognition system
- 8:31Code implementation steps
- 11:44Training the gesture model
- 14:14Making predictions with the model
- 15:20Live demo of the system
- 16:11Conclusion and future possibilities
related talks
28mBuilding Motion Controlled Websites in JavaScript
22mStrike a Pose: Gesture Recognition in JS with Machine Learning
51mRealTime Exoskeleton-Mapping and Body Segmentation on the Browser with Tensorflow.js
05mIncredible things your community is creating with TensorFlow.js
30mBuilding Machine Learning Models on Edge with React
32mSelf-Portraits in JavaScript
21mHands-Free Coding With Gaze Control in JavaScript
26mSelf-Portraits in JavaScript