Can AI Disrupt Speech Compression?
summary
Neural models compress speech down to a few kilobits with much better quality than old low‑rate vocoders and competitive with standard codecs at far lower bitrates. A proof of concept uses Codec 2 + WaveNet prototype to show generative decoders can sound like ten times the bitrate. Practical work replaces WaveNet with Lyra v1 (GRU, sparse matrices, multi‑rate) and then moves to SoundStream / Lyra v2 autoencoder with GAN losses and quantizer, which runs real‑time on a phone CPU, matches Opus/EVS at 3–12 kbps and supports denoising.
AI-written summary · about
chapters (16)
- 0:00Intro
- 0:20What is speech coding
- 0:52Is there a need
- 2:12How few bits
- 3:51Types of coders
- 5:54How speech coders are used
- 6:49Neural networks
- 10:29Challenges
- 13:45Coder Lyra
- 14:50Neural Engine
- 15:24Performance
- 17:50Medium bitrate
- 18:09High bitrate
- 19:12Clean up
- 20:06Original
- 20:47Summary
related talks
29mHow AI boosted audio processing?
22mWhy TTS Models Now Look Like LLMs
27mWhat happens when AI starts grocking streaming audio directly
11mTxt2Vid: Ultra-Low Bitrate Compression of Talking-Head Videos via Text
53mGenerative AI: Overview and use cases
58mMusic Generation via Masked Acoustic Token Modeling
06mCountering Acoustic Adversarial Attacks in Microphone-equipped Smart Home Devices
01h 18mUnderstanding And Visualizing Speech Quality