Building one multitask model to rule them all
summary
A PyTorch library called Octopod streamlines training by shuffling multi-data set training batches, combining ResNet image heads with a burp text encoder and flexible ensembles. The team diagnoses destructive interference in multitask ResNets, splits ResNets by related-task groups, and recovers accuracy (single-task color 95% vs multitask 93% initially; after the fix color ~94%, pattern ~85%, season ~84%). The library is open-sourced and the approach works in practice.
AI-written summary · about
chapters (14)
- 0:00Multitask model project intro
- 1:08Project background and problem
- 3:12What is multitask learning
- 5:36Initial model architecture
- 8:00Multi-dataset training
- 9:24Adding text to model
- 11:13Codebase refactoring
- 15:06Collaboration and communication
- 17:11Adding season attribute
- 18:34Destructive interference discovery
- 20:38Solving interference in models
- 24:10Lessons from work styles
- 24:52Open sourcing Octopod
- 26:56Closing remarks
related talks
34mComputer Vision and NLP for Multi-Task Fashion Attrb. Modeling
28mIndustry NLP: Multitask & Continual Learning
01h 31mPyTorch Workflow Mastery: A Guide to Track and Optimize Model Performance
24mMulti-tasking fraud detection From Trees to MLPs
12mIntroduction to TorchMultimodal
47mA Complex Picture of Multi-task Learning
26mAccelerating Explorations in Vision and Multimodal AI Using Pytorch...
15mRefurbish Your Training Data: Reusing Partially Augmented Samples...