Talk by Om Thakkar (OpenAI)
summary
Audits target ASR memorization by inserting synthetic audio canaries and measuring exposure versus a 20,000-sample holdout using a 600M parameter model. Random-text TTS canaries barely register even at 16 inserts, but fast 4x TTS canaries produce high exposure after 1–4 occurrences. Mitigation by per-example gradient clipping removes measurable memorization but halves throughput; per-core clipping matches most privacy gains, restores throughput, and yields ~2% relative WER change while reducing exposure from ~13.5 to ~2.
AI-written summary · about
chapters (19)
- 0:00Introduction
- 0:45Privacy leakage in ML
- 3:08Attacks on speech models
- 5:11Secret Sharer Framework
- 7:31Auditing ASR challenges
- 8:43Experiment setup
- 10:12Initial results
- 13:01Fast canaries design
- 14:10Fast canaries results
- 16:35Memorization vs generalization
- 23:30Mitigating memorization
- 25:04Per-example gradient clipping
- 28:26Model utility impact
- 30:20Micro-batch clipping
- 32:50Per-core clipping benefits
- 35:09Exposure thresholds evaluation
- 37:30Future privacy directions
- 41:44Privacy upstream training
- 42:56Conclusion
related talks
27mWhat happens when AI starts grocking streaming audio directly
22mBuilding next-gen speech synthesis for Bookmate audiobook service
16mThe End of Awkward AI Transcriptions
25mBeyond Transcription: Building Voice AI That Understands Conversations
22mWhy TTS Models Now Look Like LLMs
35mTricks Learned from Scaling WhisperSpeech Models to 80k+ Hours of Speech
35mSpeech Synthesis with Tacotron2 and PyTorch
14mLightning Talk: Enabling the Audio Modality for Language Models