Beyond Supervised Learning with Zero-Shot Vision
summary
Zero-shot classification uses a joint image–text embedding space to label unseen classes by similarity rather than softmax, trained with contrastive learning with cosine similarity; OpenAI’s CLIP uses roughly 400 million image‑text pairs and open implementations train on LAION 2B and 5B datasets. Practical outcomes: text‑to‑image retrieval and lightweight zero‑shot object detection from image patches work without retraining, and transfer learning cuts labeled data needs (example cited: 10,000 normal vs 21 damaged images suffices to learn the concept).
AI-written summary · about
chapters (9)
- 0:00Zero-shot vision intro
- 1:50Supervised learning limits
- 5:40What is zero-shot learning
- 8:08Zero-shot learning with CLIP
- 14:24How CLIP works
- 17:54CLIP applications
- 21:30CLIP training datasets
- 23:17CLIP demos
- 24:47Q&A
related talks
41mLearning from multiple modalities, Predicting on unseen tasks
32mEverything you wanted to know about VLMs but were afraid to ask
26mMaximizing Multimodal: Exploring the search frontier of text-to-image models
38mPractical Few-Shot Learning
42mDALL·E and CLIP from OpenAI: a new era in machine learning
13mInside Multimodal Models
20mEasily reduce Deep Computer Vision to shallow NLP
26mFew-Shot Learning in Conversational AI