Efficient data labelling with weak supervision
summary
Claims that weak supervision both speeds labeling and reduces annotation-caused failures, with a concrete workflow and an actual tool: data qa (open source) built on Snorkel, spaCy and Elasticsearch with a React UI. Examples include 3,000 Amazon product descriptions where 32 simple rules cover about two thirds (~2,000) items, a regex rule finding 220 NER matches, and a reported class precision around 67%. A hosted commercial edition with extra ML training and PDF support exists.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 0:57Datacentric AI
- 2:23Top challenges
- 3:18Data centric AI
- 3:56Labels are hard
- 5:37Weak supervision
- 9:08Summary
- 9:57Example
- 16:18Typical approach
- 18:03Naming entity recognition
- 25:03Questions
related talks
33mWeak supervision: a new paradigm for unreliable data
37mCreating Knowledgebases from unstructured text
29mLabeling Tools are Great, but What About Quality Checks?
51mHandling the problem of hand-labeled training data with data programming...
29mAnnotating data the right way
25mData Labeling as Religious Experience
31mCreate text classifiers in a few hours using the open-source Label Sleuth system
29mGotta catch'em all recognizing sloppy work in crowdsourcing tasks