Privacy of Decentralized Machine Learning
summary
Shows that decentralization alone does not stop data reconstruction: honest-but-curious attackers that observe gossip messages can reconstruct private images and gradients on realistic graphs, sometimes recovering most nodes. Proposes a relaxed, per-neighbor network differential privacy using Gaussian DP and a matrix factorization correlated noise mechanism for decentralized SGD. Analysis ties privacy budgets to graph topology, yields nonvacuous bounds and experiments where private decentralized training approaches the nonprivate baseline, improving privacy–utility tradeoffs.
AI-written summary · about
chapters (17)
- 0:00Decentralized ML privacy intro
- 1:25Why privacy and decentralization matter
- 3:00Federated vs fully decentralized learning
- 4:10Privacy threats beyond raw leaks
- 6:33Attacking decentralized gradient descent
- 9:06Differential privacy decentralized
- 10:36Network differential privacy
- 13:49Privacy amplification via decentralization
- 17:11Private random walk algorithm
- 19:33Graph topology and privacy
- 20:20Matrix factorization for noise
- 23:12Decentralized DP-SGD with noise
- 25:19Privacy analysis for non-square matrices
- 26:52Privacy-utility trade-offs
- 27:36Optimizing noise correlation
- 28:03Conclusion and future directions
- 28:51Q&A
related talks
35mPairwise Network Differential Privacy
14mDistributed Private Aggregation in Graph Neural Networks
31mScaling privacy embedded distributed ML systems
01h 14mTutorial: Privacy, Part II
21m"SECRETS ARE LIES, SHARING IS CARING, PRIVACY IS THEFT." - A Dive into Privacy Preserving
31mIs your model private?
48mSynthetic Data as an Enabler for Learning from Decentralized Private Data
33mHeterogeneity and Privacy in Modern Learning