The next level apps for Web developers Machine learning on browsers 🤯
summary
Shows concrete browser ML pipelines: load TensorFlow.js in-page, feed 60 FPS webcam frames to landmark models that return X/Y arrays and confidence scores, then map those arrays with fingerpose or custom math to detect gestures. Demonstrations include training a model in the browser (no backend) to control Pac-Man with face movements and using phone accelerometer JSON to teach a Duck Hunt controller. Notes performance tools: web workers, web streams and WebGPU, plus quick prototyping with teachablemachine.withgoogle.com.
AI-written summary · about
chapters (18)
- 0:00Introduction and motivation
- 1:07What is machine learning
- 2:11TensorFlow.js basics
- 3:35Real-time hand tracking demo
- 5:22Custom interactions with math
- 6:48Gesture recognition abstraction
- 7:54Pre-trained models in browsers
- 9:17Training a game with face commands
- 10:24Training a phone-controlled game
- 12:50Teachable Machine and prototyping
- 13:34Browsers as powerful ML platforms
- 15:17Roles of data scientists vs developers
- 16:20Google dinosaur AI project
- 17:47Interactive Q&A bots
- 18:49Use cases in daily life
- 20:40Audio translation bots
- 21:43Industrial and automation ideas
- 22:46Challenge and next steps
related talks
29mSuperpowers for next gen web apps: Machine learning
15mThe future of machine learning using JavaScript
24mWeb Apps of the Future With Web AI
15mMachine learning for next gen web apps with TensorFlow.js
19mML at Client Side
21mMachine Learning & JS Take your PWA to the next level
34mThe Browser, The Machine Learning and The Future we might not have imagined just less than a year ago
44mArtificial Intelligence and Machine Learning in the browser