Data Curation: Transforming Bytes into AI Gold
summary
Claim: more than 90% of production effort goes into cleaning and curating data, not model code. Describes using ultra‑high‑resolution aerial imagery (100M+ images) and geo‑fencing species rules to catch out‑of‑distribution errors (kangaroo in UAE example), using SD‑dup achieves same performance with 50% data and FineWeb filtering to cut compute 10x, synthetic scenes from game engines for edge cases, warnings about catastrophic forgetting, and practical tool use like metaflow for artifact tracking and standard ML stacks (PyTorch/JAX).
AI-written summary · about
chapters (7)
- 0:00Introduction to data curation
- 1:50Data as code in machine learning
- 3:17Challenge one out of distribution data
- 6:09Challenge two data biases
- 10:47Challenge three data quality vs quantity
- 15:56Key takeaways on data curation
- 16:41Q&A session
related talks
30mData-Centric AI Cookbook-let's prep that data
11mData-centric AI: A new paradigm
21mDataEngBytes 2023 - SYD-T2-04 - Suneeta Mall
54mHow to handle the luxury of having too much training data
34mGenerative AI Model Data Pre-Training on Kubernetes: A Use Case Study
41mModels Schm-odels: Why you Should Care about Data-Centric AI
14mData Lifecycle Management as Foundational Pillar to AI Agents
46mPath of Data & AI Challenges