Privacy Amplification by Sampling in Practice
summary
Demonstrates how truncated Poisson sampling lets JAX-style JIT pipelines keep fixed batch sizes by padding or truncating Poisson-drawn batches, trading rare truncation for privacy loss and recovering up to ~40% more effective batch use versus naive fixed-subset sampling. Shows correlated-noise DP matrix factorization that cancels noise across rounds to match or beat standard DP-SGD utility, and a practical Monte Carlo accounting implementation that is embarrassingly parallel but needs O(log(1/δ)/δ) samples for tight δ guarantees.
AI-written summary · about
chapters (12)
- 0:00Simons Institute intro
- 1:44Privacy and ML focus
- 4:11Differential privacy overview
- 9:04Privacy amplification by sampling
- 11:08Fixed batch size challenges in JAX
- 13:34Truncated Poisson sampling
- 22:22Matrix factorization privacy
- 25:51Federated computation threats
- 31:46Balls and bins sampling
- 33:52Monte Carlo accounting
- 38:00Combining matrix factorization and balls and bins
- 40:29Summary and open problems
related talks
50mPrivacy Amplification by Sampling and Renyi Differential Privacy
01h 14mTutorial: Privacy, Part III
54mPrivacy Amplification from Structured Algorithmic Randomness
17mPrivate Tuning of LLMs in Practice: From VaultGemma to Custom Fine-Tuning
01h 14mTutorial: Privacy, Part II
47mBlack-box Random Differential Privacy with Sensitivity Sampling
39mPufferfish Privacy Mechanisms for Correlated Data
31mShort Industry Talks pt. 1