Protecting sensitive data in huge datasets: Cloud tools you can use
summary
Shows concrete tools and numbers: uses Cloud DLP API (streaming mode, 90+ classifiers, image redaction, crypto‑replace) and BigQuery to scan and transform massive datasets. Demonstrates on a 170M taxi trip plot, a 240M rows wiki dataset (290 GB) and Mexico births (≈13M records), computes k‑anonymity, l‑diversity, k‑map and delta‑presence, and applies masking, tokenization, date‑shifting and aggregation. Includes automated detection rules, custom regex/dictionaries, and examples of how much data is removed or generalized to reach target risk metrics.
AI-written summary · about
chapters (35)
- 0:00Introduction
- 3:31Running your own predictions
- 8:28Privacy and open data
- 11:54Chat bot example
- 12:52Cloud DLP
- 13:13Can you use it in your pipeline
- 13:39At least 90 classifiers
- 14:11Transform data
- 14:31Mask data
- 14:43Identify risk
- 15:39Date shifting
- 16:19Custom detection
- 17:12Dictionaries
- 17:26Tokenization
- 17:50Encryption
- 21:15What is BigQuery
- 26:01What do you do with this data
- 27:03Definitions
- 27:58Mission measures
- 28:15K anonymity
- 28:36Example Mexico
- 31:53Quasi Identifiers
- 35:26Automate
- 35:41Measuring key anonymity
- 36:48Eldiversity
- 37:10Anonymity
- 38:07Kmap anonymity
- 41:12Delta presence
- 41:59Key anonymity
- 42:36Best practices
- 44:19Public datasets
- 44:41Sharing data
- 44:54Differential privacy
- 45:07Understanding your data
- 45:33Contact Felipe
related talks
01h 59mDeep Dive Workshop – Data Security and Data Protection
18mData Mapping at a Billion Dollar Self-Driving Vehicle Startup
31mLeverage AI to Protect Your Data Assets... from AI
01h 59mDeep Dive Workshop – Data Security and Data Protection
37mDon’t Worry About BigQuery PII
39mData Privacy Engineering to power Responsible Enterprise AI
48mCatch Me if You Can: Hunting Cloud Exfiltration Using Anomaly Detection
28mOpen the gate a little: strategies to protect and share data