Acoustic Word Embeddings for Low Resource Speech Processing
summary
Proposes that variable-duration speech segments map to fixed vectors so repeated words can be found without transcripts, using acoustic word embeddings trained on random segmentations, clustered with a Bayesian Gaussian mixture model, and optimized via Gibbs sampling plus dynamic programming for boundary search. Pipeline iterates: embed → cluster → resegment → reembed. Demonstrates useful keyword spotting on low-resource corpora and a Uganda radio use case; performs well for single speakers but degrades with multi-speaker, tonal languages and heavy code-switching.
AI-written summary · about
chapters (21)
- 0:00Introduction
- 3:45Hermans background
- 4:52TTI Chicago
- 5:48Low Resource Speech Processing
- 11:46Let a Model Learn
- 14:21Cognitive Science
- 17:55Zero Resource Speech Recognition
- 21:46Recap
- 24:36Window sizing
- 28:02Language optimization
- 33:15Code switching
- 34:38Representing phonemes
- 38:38Our approach
- 40:17Why is this useful
- 42:48Acoustic word embeddings
- 45:44Optimizing the embedding space
- 46:47clustering within the embedding space
- 49:42word boundaries
- 51:33score
- 52:38Gibbs sampling
- 57:15Final thoughts
related talks
28mEmbeddings
56mRandom Embeddings, Matrix-valued Kernels and Deep Learning
10mUtilizing Word Embeddings and Gradient Boosting
28m"Embeddings: How Computers Learned to Read"
31mTaking AI/ML to the next level: Text!
01h 03mDrop the Bass with Embedding and Vectors in Azure AI Search
39mSemi-Supervised Bootstrapping of Relationship Extractors
25mLooking at Sound