Vision-Language Models for Fraud Detection
summary
Fine tuned a Florence VLM on internal ID images to create a single “universal document expert” that replaces dozens of specialized networks, training on up to 17 tasks simultaneously without measurable capacity loss. Results match current multi-model production quality on most tasks, but fail on three machine readable zone cases and exotic languages. Production inference packs image tokens and questions, uses only the first LLM token to decide, achieving about 46 ms inference on high end hardware. Edge use and unseen text patterns remain problematic.
AI-written summary · about
chapters (12)
- 0:00Vision-language models intro
- 1:07Incode identity verification
- 1:53ID fraud detection challenges
- 2:57Why vision-language models
- 4:04Training universal document model
- 6:12Model vs production performance
- 7:38Model limits and task formulation
- 10:52Efficient inference
- 13:21Batch processing optimization
- 15:05Vision-language limitations
- 18:45Summary and future research
- 19:51Q&A
related talks
32mEverything you wanted to know about VLMs but were afraid to ask
35mGenerative AI for Fraud Detection & Claims Analysis
01h 08mMastering Visual AI with Vision-Language Models & Advanced Evaluation Techniques
32mTeaching Machines to See:Vision-Language Models Made Efficient with SmolVLM
51mVisual Quality Assurance Using Vision-Language Models
17mVision AI in 2025
01h 23mMultimodal Deep Learning in the Real World
40mKeynote: Open-source Multimodal AI in the Wild