Visualization Question Answering Using Introspective Program Synthesis
summary
POLL refines a VQA model’s noisy outputs by synthesizing executable explanation programs: a top-down abstract synthesis stage emits partial programs and a second optimal synthesis stage fills holes via a MaxSMT encoding (MaxSMT) that scores triangle alignments between visualization elements, query tokens, and program productions. Built on TAPAS, it implements a ~20-node DSL, runs on 629 VQA benchmarks, and raises solved cases by about 23% (up to ~27–28% under wider settings); source code is available on GitHub.
AI-written summary · about
chapters (6)
- 0:00Introduction
- 1:13Visualization Question Answering
- 10:23Motivation Example
- 13:03Abstract Synthesis Module
- 14:36Optimal Synthesis Module
- 16:14Evaluation
related talks
16mType-Directed Synthesis of Visualizations from Natural Language Queries
51mVisual Quality Assurance Using Vision-Language Models
22mVajra: step-by-step programming with natural language
16mImproving Oracle-Guided Inductive Synthesis by Efficient Question Selection
14mTunneling Through the Hill: Multi-Way Intersection for Version-Space Algebras in(…)
24mInferencing underspecified natural language utterances in visual analysis
32mBeyond Vibe Coding: Leveraging Foundation Models to Infer Intent
26mDSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines