Greenfield vs. Brownfield Data Labeling
summary
Distinguishes greenfield (fast prototypes, focus on quantity) from brownfield (improve existing labels, focus on quality) and gives concrete methods: use weak supervision with labeling functions, zero‑shot Hugging Face classifiers, distance supervision and crowd vs in‑house tradeoffs to generate bootstrapped training sets; use confident learning to estimate label error rates (example estimating ~25% noisy labels) and prioritize corrections; use neural search with embeddings and diversity sampling to find outliers and representative examples; mentions inter‑annotator agreement metrics and an upcoming open‑source tool.
AI-written summary · about
chapters (20)
- 0:00Data labeling concepts
- 1:11Greenfield vs brownfield labeling
- 2:38Greenfield labeling strategies
- 4:01Weak supervision for labeling
- 6:52Heuristics in weak supervision
- 7:13Labeling vs model training trade-offs
- 8:35Manual labeling for validation
- 9:18Neural search for labeling
- 10:21Diversity sampling for outliers
- 11:24Neural search for representative data
- 12:25Automating labeling for PoC
- 13:09Brownfield labeling for quality
- 13:51Confident learning for label errors
- 14:56Error rates in training data
- 15:41Debugging brownfield heuristics
- 16:44Inter-annotator agreement
- 17:46Technologies for greenfield and brownfield
- 18:07Future of data labeling
- 19:32Open-source labeling tools
- 20:15Q&A
related talks
33mWeak supervision: a new paradigm for unreliable data
42mClean Labels for ML
36mProgrammatically creating and managing training data
39mThe practice of acquiring good labels
29mLabeling against all odds
29mLabeling Tools are Great, but What About Quality Checks?
21mDataEngBytes 2023 - SYD-T2-04 - Suneeta Mall
02mDBTB INT Omar Alonso