Designing a Private Logging Pipeline
summary
Mobile logging faces battery and bandwidth limits and must protect data from billions of users while still enabling debugging and metrics; the pipeline uses on-device annotations (AOSP-style) plus automatic string scrubbing and enum-based exception logging to avoid PII, rotates or removes identifiers and can replace per-device IDs with a one-time ping for MAU/DAU. Server-side techniques include encrypt-shuffle-analyze (used on Fuchsia), random sampling, cross-device aggregation and central DP to scale to billions of devices and trillions of log lines; anonymization also shrinks datasets and cuts processing costs.
AI-written summary · about
chapters (9)
- 0:00Private logging pipelines intro
- 1:13Mobile logging differences
- 2:15Minimizing device data
- 3:38Sensitive strings handling
- 5:23Server-side privacy enforcement
- 6:46Policy-based access retention
- 7:47Server-side privacy solutions
- 9:50Future privacy techniques
- 11:39Scalable privacy benefits
related talks
29mApplication logging in the era of GDPR
15mAnonymization Aspects of a Low-latency VoIP Security Analytics System
40mKeeping data private in real-time pipelines
37mA Needle in a Haystack: How to Find a Threat Hidden in Over 6 Billion Logs Per Day
16mBuilding an End-to-End De-Identification Pipeline for Advertising Activity Data at...
26mSecuring Containers: Preventing Data Leaks
37mHey, You, Get off My Private Data: Do Apps Respect Your Privacy as They Claim?
38mBehind the Firewall: Designing Secure LLM Pipelines for Regulated Data