Active learning in the interactive python environment
summary
Claims active learning cuts labels roughly in half — e.g. reduced labels from 10,000 to 4,000 and 7,600 to 3,600 on image tasks. Implementation uses Jupyter + ipywidgets + scikit-learn with a PostgreSQL queue, Docker/voila deployment and an orchestrator that retrains periodically (every 30s when 10 new labels). Uncertainty methods include entropy margins, Monte Carlo dropout and SGD-sampling for approximate posteriors. A notebook-backed labeling UI called superintendent is built and deployed; it scales to multiple labelers and has been used successfully on image and time-series projects.
AI-written summary · about
chapters (8)
- 0:00Active learning in Python
- 1:53Labeling data essentials
- 4:44Active learning efficiency
- 10:21Ranking with model uncertainty
- 15:14Active learning with superintendent
- 22:02Scaling distributed workflows
- 29:47Real-world applications
- 30:28Q&A
related talks
01h 31mInteractive Learning of Classifiers and Other Structures
30mActive Learning and Human-in-the-Loop Machine Learning
37mActive Teaching, Human Learning
51mEfficient Deep Learning with Humans in the Loop
18mActive Learning for Neurosymbolic Program Synthesis
45mActive learning and weak supervision in NLP projects
04mLabel Studio
58mLabeling a Data Set Using Sublinearly Many Queries