Tutorial: Privacy, Part III
summary
DP-SGD suffers from realistic sampling gaps and amplification limits, so the lecture reframes iterative training as continual observation and private release of prefix sums. Using the binary tree mechanism adds only O(log T) noise per prefix instead of O(T), and the factorization or correlated noise framework trades column/row norms of matrices C and B for privacy and utility. Practical improvements (Toeplitz, banded, BLT) push utility toward the 1/(pi) ln n bound while keeping time and memory near D log n.
AI-written summary · about
chapters (10)
- 0:00DP-SGD introduction and challenges
- 3:25Continual observation perspective
- 11:47Differentially private online learning
- 16:40Prefix sums and factorization mechanisms
- 30:14Optimizing factorization mechanisms
- 44:25Practical DP learning considerations
- 51:39High-dimensional private estimation
- 53:06Federated learning privacy challenges
- 1:02:30Privacy parameter limitations
- 1:04:26Future privacy research directions
related talks
30mLarge Scale Private Learning on Data Streams, and the BLTs
43mPrivacy Amplification by Sampling in Practice
01h 07mLarge-Scale Private Learning, Part II
31mShort Industry Talks pt. 1
19mPractical Considerations for Differential Privacy
01h 02mLarge-Scale Private Learning, Part I
26mAaron Roth, Associate Professor, University of Pennsylvania, NYC 2017
01h 14mStatistical Inference and Privacy, Part III