Visual Literacy: Complex Document Retrieval with VLMs
summary
Claims that vision‑language embeddings close the “visual information gap” for charts, tables and complex layouts by using a 3B‑parameter Quen 2.5VL backbone and task adapters to produce Gina embeddings v4; training uses contrastive loss with hard negatives, matrioska truncation and quantization‑aware steps, and optimizes both single (248‑dim) and multi (128‑dim token) vectors. A new GIA VDR benchmark spans ~95 tasks in 20 languages and shows VLM embeddings beat OCR‑only and CLIP baselines, with multi‑vector retrieval giving the best recall.
AI-written summary · about
chapters (7)
- 0:00Introduction to talk
- 0:44Basics of document retrieval
- 3:57Challenges with complex documents
- 10:25Vision language models as solution
- 18:15Introducing Gina embeddings v4
- 21:07Benchmarking complex retrieval
- 29:39Q&A session
related talks
01h 08mMastering Visual AI with Vision-Language Models & Advanced Evaluation Techniques
23mAVA: Towards Agentic Video Analytics with Vision Language Models
28mSupercharge Information Extraction with Vision-Based LLMs
17mVision AI in 2025
01h 26mBuilding an Image Search Engine with Generative Computer Vision Models
11mPaliGemma – Making Gemma 2 see by adding a vision encoder
21mVizLens: A Robust and Interactive Screen Reader for Interfaces in the Real World
01h 18mComputer Vision after the Victory of Data