Towards Lifecycle Unlearning Commitment Management: Measuring Sample-level...
summary
Proposes Interpolated Approximate Measurement (IAM), a sample-level unlearning metric that interpolates model outputs via a bounded logit map and fits those interpolations to a Gaussian to score unlearning completeness; requires only one pre-trained shadow model, scales to Llama 27B, and achieves near-perfect binary membership AUC plus strong correlation with approximate unlearning. Includes theoretical analysis of the scoring efficiency, uses moment estimators for parameters, benchmarks seven approximate-unlearning methods, and reveals systematic over-unlearning and under-unlearning risks; code is on GitHub.
AI-written summary · about
chapters (10)
- 0:00Machine learning commitments
- 1:15Why commitments matter
- 2:41Limitations of ML methods
- 4:24Auto Responses framework
- 6:32Binary to spectrum inference
- 7:59Interpolated Approximate Measurement
- 10:10Evaluating AM on training
- 10:51Testing AM on large models
- 11:34Benchmarking opposing learning
- 12:18Conclusion and findings
related talks
23mHow can a learnt ML model unlearn something: Framework for "Machine Unlearning"
15mLearning and Unlearning Your Data in Federated Settings
46mIncreasing Participant Agency by Understanding the Impact of Training Data
25mConfidence-Based Active Learning
22mHow to Detect Silent Failures in ML Models
54mLearning Uninformative Representations
37mEfficient Algorithms for Reliable Machine Learning
56mDehumanizing Agents: Why Explainability is Crucial in the LLM Era