Building Motion Controlled Websites in JavaScript
summary
Explains using TensorFlow.js models PoseNet, FaceMesh, HandPose with getUserMedia and a WebGL backend, calling estimateSinglePose/estimateFaces/estimateHands to read keypoints; FaceMesh is ~3 MB and HandPose ~12 MB, both top out near 40 FPS. Shows working demos: BitPose (a Beat Saber clone controlled by wrist positions) and FaceScroll (head-tilt scrolling using Z coordinates). Notes limits: webcam lacks true depth, lighting and latency reduce accuracy, and model size/performance tradeoffs matter when deploying.
AI-written summary · about
chapters (11)
- 0:00Start
- 2:18Motion control
- 5:05Pose detection
- 6:42Tools
- 8:50Demo
- 10:42Accessibility
- 12:37New interactions
- 13:35Health
- 15:00Performance
- 16:18Accuracy
- 16:57User Experience
related talks
19mHand gesture-controlled robot through the Internet
42mExploring Alternative Interactions in JavaScript
54mDeep Learning in JS
26mSelf-Portraits in JavaScript
29mSuperpowers for next gen web apps: Machine learning
41mTensorFlow.js 101: ML in the Browser and Beyond
05mIncredible things your community is creating with TensorFlow.js
32mWe Also Can Do It! Machine Learning in Javascript!