Still using one hot encoding? Here’s some ‘HELP’!
summary
Proposal replaces one-hot labels with dense, lower-dimensional prototypes computed by equiangular Grassmannian line packing, trains networks as regression targets using cosine regression targets (replace categorical cross-entropy), and does inference by nearest-neighbor search in angular space with FAISS and RAPIDS on GPU. Claims: ImageNet-1k prototypes concentrate near 86.5–90° in 512 dims; full-ImageNet inference runs in ~0.6s on a commodity GPU versus ~25 minutes with KD-tree (≈103× speedup). Reports out-of-the-box or slight accuracy gains on vision, SST sentiment, time series and speech when only the last layer is retrained.
AI-written summary · about
chapters (14)
- 0:00Start
- 1:15Talk Organization
- 2:11Stubborn persistence of bad ideas
- 3:28One-hot encoding
- 3:47Available by default in all scientific computing frameworks
- 9:51Information theoretical label embeddings for large-scale image classification
- 10:03Angular Visual Hardness
- 12:48A brief tour
- 13:36Algorithm for equi-angular line packings
- 20:25Section 5
- 23:24Glove embeddings for ImageNet -1K dataset
- 24:03Proposing a new architecture for the SoTA dataset?
- 25:01Call for contributions: A bridge between the two-worlds
- 26:21Questions?
related talks
56mTutorial on Deep Learning II
01h 04mRandomized Embeddings and Neural Networks
28mNext-generation frameworks for Large-scale Machine Learning
17mOptimized Image Classification on the Cheap
01h 30mDeep Learning Techniques for Monocular Depth Estimation
42mDeep Learning for Computer Vision
40mVisual concept learning from few images
51mTips & Tricks for Fast Neural Net Inference in Production