Evaluating the Usability of Differential Privacy Tools with Data Practitioners
summary
Usability study tests four Python DP tools (DiffPrivLib, Tumult Analytics, PipelineDP, OpenDP) with 24 US data practitioners (12 novices, 12 experts) on count, sum, and mean tasks. Novices improve conceptual DP knowledge but many implementations fail: all six participants violate DP in tasks 2 and 3 with DiffPrivLib/PipelineDP by ignoring sensitivity or misusing APIs. Tumult Analytics has the best balance of completion and correctness, while OpenDP prevents violations but yields low completion and satisfaction. Recommendations: strict error checks, clearer APIs, searchable examples, and actionable error messages.
AI-written summary · about
chapters (7)
- 0:00Differential privacy usability study
- 2:16Study goals and tools
- 3:44Participant selection and design
- 5:31Learning outcomes and task completion
- 7:37DP violations and API issues
- 9:46Balancing DP and usability
- 11:30Tool improvements and docs
related talks
19mPractical Considerations for Differential Privacy
09mExpanding Differentially Private Solutions: A Python Case Study
17mPrivacy in Theory, Bugs in Practice: Grey-Box Testing for Differential Privacy Libraries
47mOpen DP: A Proposal for an Open-Source Suite of Differential Privacy Tools
01h 00mPANEL: Statistical Theory, Privacy and Data Analysis
35mDifferential Privacy Made Practical
01h 01mFrom Theory to Practice: Crafting Differential Privacy Systems with Haskell
45mMaking Privacy Technology Accessible: Benchmarks and Platforms