Design for Data
summary
Argues that data generating process and workflow, not algorithms, determine success. Shows Housing.com reaching million properties but supply curves miss user demand, so an API plus incentive change fixes collector bias. Shows clickstream tracking break when UI changes. Describes PaySense using average monthly bank balance from SMS feeds, grappling with sparse spans and interpolation, and using SMS salary matches with a 61% error-cluster versus bias-cluster model to detect exaggeration. Offers reproducibility → replicability → robustness checklist.
AI-written summary · about
chapters (8)
- 0:00Start
- 6:04What Is the Data Generating Process
- 6:39Sampling Strategies
- 14:20Event Data Modelling
- 16:39Data Is like the New Oil
- 17:57Average Monthly Bank Balance
- 29:36Mixed Distributions Approach
- 31:27The Curse of Knowledge
related talks
46mWhat's the Science in Data Science?
36mArchitecting Data: A Deep Dive Into the world of Synthetic Data
27mData Creation: How Organizations Are Deliberately Creating Data To Power ML & Data Applications
40mData and Algorithmic Bias in the Web
43mDomain-driven Data Science
34mData Science beasts (failures) and where to find them
44mKeep it Clean: Why Bad Data Ruins Projects and How to Fix it
37mBehind the AI Curtain - Designing for Machine Learning Products