Sensing to Hear: Speech Enhancement for Mobile Devices Using Acoustic Signals
summary
WaveVoice uses the phone speaker to emit inaudible multi-frequency acoustic signals and captures lip motion via reflections, fusing that with noisy audio in a dual-CNN encoder-decoder (separate magnitude and phase subnets) to output enhanced speech. An Android app collects a dataset from 25 volunteers; a self-supervised personalization method reaches validation-level performance after ~200 per-user samples. Offline and online tests show notable SDR and PESQ gains versus audio-only baselines and lower character error rate on speech recognition.
AI-written summary · about
related talks
36mVoice Enabling Mobile Apps and Custom Hardware Devices
20mLip-Interact: Improving Mobile Device Interaction with Silent Speech Commands
35mHaving a Voice on Android: How hard can it be?
21mDeveloping Machine Learning Based Speech Enhancement Models for Teams and Skype
12mDirection-of-Voice (DoV) Estimation for Intuitive Speech Interaction with Smart Device Ecosystems
45mAuditory Intelligence : beyond speech recognition
22mOn-Device Speech Recognition for Mobile Devices and Custom Hardware
22mUbicoustics: Plug-and-Play Acoustic Activity Recognition