AVA: Towards Agentic Video Analytics with Vision Language Models
summary
AVA builds an Event Knowledge Graphs index from long streams by windowed captioning with a 7B VLM (e.g., ChainOne VL), semantic merging via BLEU/BERT-like scores, and temporal/entity edges, then runs an agentic retrieval-generation loop where an LLM explores the EKG with forward/backward/requery/summarize actions. Results: 62.3% on LVBench, 64.1% on VideoMME-Long, 75.8% on AVA-100 (8 videos >10 hours, 120 QA). Indexing runs >2 FPS; agentic search dominates time (~80%). System improves reasoning by ~40 points and yields up to ~20% QA gains.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 2:25Video analytics intelligence levels
- 6:00AVA’s L4 reasoning approach
- 7:47Open-ended video analytics challenges
- 9:55Efficient ultra-long video handling
- 12:04Event Knowledge Graph
- 15:37Agentic retrieval and evidence gathering
- 18:52Consistency-based answer selection
- 19:12Long-video benchmark evaluation
- 22:23Summary and future directions
related talks
35mLarge-Scale Video Intelligence
37mVisual Literacy: Complex Document Retrieval with VLMs
51mVisual Quality Assurance Using Vision-Language Models
15mVideo Analytics with Zero-streaming Cameras
27mThe Cutting Edge of AI: From LLMs to Multimodal Intelligence
32mVideo Action Models
01h 08mMastering Visual AI with Vision-Language Models & Advanced Evaluation Techniques
18mCoVA: Exploiting Compressed-Domain Analysis to Accelerate Video Analytics