Synthetic data: when, why, and how
summary
Argues synthetic tabular data is practical for privacy, load testing, and rare-event simulation, showing concrete techniques: sampling marginals, sampling joint distributions for start/end station pairs (3.47M possible pairs, 8,600 pairs account for 25% of trips), and fixing obvious invariants (end time after start, station name lookup). Uses XGBoost F1=1 to detect mismatches between real and naive synthetic data, shows simple fixes break that classifier, and points to t‑digest, VAEs/GANs, TabFormer, PyMC/Stan and differential privacy for next steps.
AI-written summary · about
chapters (20)
- 0:00Introduction
- 2:17Evaluating query performance
- 6:33Evaluating ML pipelines
- 8:47Commercial use restrictions
- 15:19Applications
- 16:32Counterfactuals
- 17:09Similarity
- 17:57Datasaurus
- 19:16Simpsons Paradox
- 19:42Case Study
- 23:06Generating Synthetic Data
- 23:42Distributions
- 25:50Extreme values
- 28:19Training a model
- 31:20Sampling from joint distributions
- 34:10Time modeling
- 34:41Cumulative distributions
- 36:29Simulation
- 38:46Using Synthetic Data
- 39:04Recap
related talks
20mSynthetic Data: The new Super Power for AI
30m🤯 No data? No problem! Synthetic data to the rescue
54mOpen Data vs. Privacy - How Synthetic Data Can Democratize AI Innovation
13mDPSynth: From Research to Production—Engineering Differentially Private Synthetic...
06m"Improving Accuracy of AI-based Applications"
31mLife cycle of data becoming a model
09mGenerating Synthetic Data with AI
26mUnlocking the Power of Data-Centric AI.