Type-Directed Synthesis of Visualizations from Natural Language Queries
summary
Graphy converts a natural-language query and a table into a ranked set of visualization programs by parsing intents and slots with a fine-tuned BERT into refinement type specifications and then running type-directed program synthesis plus data-transformation search. It evaluates on NLVCorpus (≈700 queries over Cars, Movies, Superstore) and reports average end-to-end latency 5.89s, a 2.1× speedup from lemma learning and ~298× pruning speedup from refinement-type checks, and higher top‑10 accuracy than rule‑based and transformer baselines.
AI-written summary · about
chapters (12)
- 0:00Graphy introduction
- 2:07Neurosymbolic approach
- 3:10Type systems for visualization
- 4:34Refinement types for constraints
- 6:18Natural language to specs
- 7:46Type-directed synthesis
- 9:55Incremental synthesis with Lemma
- 11:41Evaluation setup
- 13:27Performance improvements
- 13:49Summary and open-source
- 14:10Q&A: Polymorphism in types
- 14:54Q&A: Graphy output range
related talks
20mDataTone: Managing Ambiguity in Natural Language Interfaces for Data Visualization
18mGraphScape: A Model for Automated Reasoning about Visualization Similarity and Sequencing
24mInferencing underspecified natural language utterances in visual analysis
20mEviza: A Natural Language Interface for Visual Analysis
34mSearching for My Next Chart
01h 40mKnowledge Graphs & GraphRAG: Techniques for Building Effective GenAI Applications
01h 41mGraph Intelligence: Enhance Reasoning and Retrieval Using Graph Analytics
27mData Prep for Graphs