Ubicoustics: Plug-and-Play Acoustic Activity Recognition
summary
A real-time acoustic activity recognizer tunes a pretrained AudioSet/VGG16 model using professional sound effect libraries and aggressive augmentation (amplitude, reverb via measured impulse responses, mixing) to synthesize training data. It targets 30 everyday classes and is evaluated across 50 locations, 12 users and 7 device types; results reach 93.9% hold-out / 89.6% in-the-wild and 80.4% with 20% unknowns. The system runs on edge hardware (Raspberry Pi AIY) and matches human baselines.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 1:35Related work
- 3:25Why sound effects
- 5:33How it works
- 8:14Demo
- 12:05Results
- 14:26Limitations
- 15:36Applications
- 17:21Questions
related talks
20mExploring the hidden potential of sound data
07mOk Google, What Am I Doing? Acoustic Activity Recognition Bounded by Conversational Assistant ...
01h 14mExploring the hidden potential of sound data
45mAuditory Intelligence : beyond speech recognition
25mLooking at Sound
06mSensing to Hear: Speech Enhancement for Mobile Devices Using Acoustic Signals
58mMusic Generation via Masked Acoustic Token Modeling
39mOverview and Marine Bioacoustics and Perch Model