Exploring the hidden potential of sound data
summary
Claims that everyday microphones plus Web Audio API and TensorFlow.js can do acoustic activity recognition, not word spotting but classifying actions (cutlery, coughing, typing). Demonstrates a browser demo trained with Teachable Machine and the speech-commands model, uses spectrograms as features, transfers models recorded in different environments, and recognizes several activities reliably though a new toothbrush failed. Notes practical limits: lots of samples, large model files, and weak handling of simultaneous activities.
AI-written summary · about
chapters (13)
- 0:00Start
- 2:21Microphones
- 3:22Visualizations
- 3:48The Frequency Bar Graph
- 4:09A Spectrogram
- 5:40Machine Learning
- 5:56Teachable Machine Experiment
- 10:54Live Demo
- 13:57Benefits
- 15:00Motion Sensors
- 15:48Interactive Storytelling
- 16:38Health Tracking
- 17:16Model File Size
related talks
01h 14mExploring the hidden potential of sound data
22mUbicoustics: Plug-and-Play Acoustic Activity Recognition
07mOk Google, What Am I Doing? Acoustic Activity Recognition Bounded by Conversational Assistant ...
28mHow to use Deep Learning to process audio data and classify sounds.
32mAfter Censorship, Self-Censorship… but This Time It’s Funny, ...
18mExploring alternative interactions in JavaScript
45mThe Sound of Your Secrets: Teaching Your Model to Spy So You Can Learn to Defend
38mIntegrating AI and Browser API Voice Recognition on Web App