Doing Data: The Critical Process of Data Preparation (...)
summary
A practical playbook for making ML projects work: preserve raw inputs (one lab kept blood samples for 30 years and later found genetic signals), verify labels before modeling, and explore data visually to avoid false alarms — a European provider finds missing-region drops that averaged away in three days. Build features from domain behavior (switch from PIN to signature, gas-station probes, surprising pizza signal) and track everything with Apache Drill for exploration and MapR volumes snapshots for data versioning. Add automated data unit tests to catch drifting inputs.
AI-written summary · about
chapters (20)
- 0:00Introduction
- 1:23Powerful techniques
- 2:22Blood samples
- 3:28Data preservation
- 6:05Data scientists
- 6:58Dont do that
- 8:22Kago Contest
- 10:21Data Preparation
- 16:49Data Exploration
- 21:16Features
- 26:28Fraud
- 29:41Augment Data
- 30:26Velocity
- 32:02Cross Validation
- 33:08Document
- 34:12iterative process
- 36:07Webinar
- 37:30Unit Testing
- 38:33Unit Data Training
- 39:05Women in Big Data
related talks
27mData Creation: How Organizations Are Deliberately Creating Data To Power ML & Data Applications
18mData quality: the key to long term happiness
42mTesting a Data Science Model
36mApplied Machine Learning Playbook: From Idea to Production
30mData-Centric AI Cookbook-let's prep that data
31mLife cycle of data becoming a model
28mDataEngBytes 2023 - PER-T1-05 - Ellie Gholami
27mA Look Inside Applied Machine Learning