Learning the Privacy-Utility Trade-off with Bayesian Optimization
Talkat Data Privacy: From Foundations to Applications (Simons Institute workshop, 2019) · 2019-03-04..03-08·
summary
Implementation uses multi-objective Bayesian optimization with a Gaussian process surrogate and a hypervolume probability of improvement acquisition to map error versus ε for configurable DP algorithms. Experiments include a modified sparse vector example (closed-form privacy oracle), DP-SGD on Adult and MNIST/EMNIST architectures, and comparisons to random sampling, showing BO finds Pareto-efficient hyperparameters far faster. Notes oracle evaluations can leak privacy and suggests public-data evaluation or privatized noisy evaluations as mitigations.
AI-written summary · about
chapters (14)
- 0:00Intro
- 3:19Theory vs Practice
- 5:47Example: DP-SGD
- 9:11Privacy-Utility Pareto Front
- 11:37Problem Formulation
- 15:08Pareto-Optimal Points
- 16:59Bayesian Optimization (BO)
- 19:51BO: 1-Dimensional Example
- 23:27The DPareto Algorithm
- 27:43Example: Sparse Vector Technique
- 32:44Implementing the Oracles
- 35:43Machine Learning Experiments
- 37:30DPareto vs Random Sampling
- 39:36Conclusion
related talks
28mBayesian Optimization- Fundamentals, Implementation, and Practice
24mUsing Bayesian Optimization to Tune Machine Learning Models
34mBayesian Optimization and it's application to Neural Networks
01h 14mTutorial: Privacy, Part III
39mBayesian optimisation with scikit-learn
47mPrivate targeting strategies
26mAaron Roth, Associate Professor, University of Pennsylvania, NYC 2017
09mFrom Robustness to Privacy and Back