Off-the-shelf HuggingFace models for audio deepfake detection
summary
Uses wav2vec 2.0 and WavLM embeddings from HuggingFace, averages frame embeddings into a single vector, and trains a simple logistic regression classifier on ASVspoof2019. Training set is ~22k samples, test ~64k; training F1 near 0.99 and test F1 around 0.97, with training EER reported ~1.24 and higher on out‑of‑distribution generators. No fine‑tuning; processing scales from a 36‑file demo to full dataset; failure modes include expressive real speech misclassified and limited generalization to new ElevenLabs/Respeecher‑style generators.
AI-written summary · about
chapters (13)
- 0:00Audio deepfake detection intro
- 1:52What are audio deepfakes
- 5:47Hugging Face audio SSL
- 7:36Deepfake detector pipeline
- 13:17Embeddings with Wave2Vec2 WavLM
- 15:52Training on ASVspoof data
- 18:01t-SNE embeddings visualization
- 20:58Classifier performance metrics
- 23:24Equal error rate thresholds
- 25:38WavLM embeddings results
- 27:03Real-world audio tests
- 27:46Challenges and future work
- 28:53Q&A data imbalance PCA
related talks
26mDeepfakes, audiofakes, and the future of media
26mDetecting Deep Fakes with Mice
51mPlaying Offense and Defense with Deepfakes
47mFighting AI with AI
58mPushing Deepfakes to the Limit-Fake video calls with AI
33mMachine Learning 2.0 with Hugging Face
59mTechnical Workshop: Sarcastically Speaking: Unlocking Multi-modal Sentiment Analysis With NLP and Facial Expressions
20mVoice fingerprinting & recognition