Training Embedding Model Resiliently for Multimodal M...
summary
Describes training production-grade embedding and classifier models for a vLLM Semantic Router, delivering a multilingual text embedding with 32K context window and 2D Matryoshka scaling plus multimodal embeddings at ~1.1B and 200M parameter sizes. Training uses native PyTorch DDP on AMD GPUs with ROCm-level tweaks (Torch Inductor, MXM low‑precision, symmetric memory, Torch Compile) to reach high GPU utilization and millisecond classification latency, and claims accuracy on par with frontier embedding models while extending long‑context and multilingual coverage.
AI-written summary · about
chapters (6)
- 0:00Semantic routing for inference
- 2:04Semantic router concept
- 4:55Text and multimodal embeddings
- 7:42Efficient multimodal training
- 8:07PyTorch on AMD GPUs
- 12:42Summary
related talks
41mSemantic Multimodal Multilingual Similarity Engine
30mRoblox's Journey to Supporting Multimodality on vLLM
16mEmpower AI: Building Smarter Solutions with Embeddings
28mLarge Multimodal Models: Understanding the State of the Art in Compute
22mMultilingual embeddings to scale NLP models to multiple...
20mOPER: Optimality-Guided Embedding Table Parallelization for Large-scale...
01h 08mGoing multi-modal
43mDemocratizing NLP content modeling with transfer learning using GPUs