Trust, but Verify: Optimistic Visualizations of Approximate Queries for Exploring Big Data
summary
Pangloss implements optimistic visualization by running fast approximate queries with Sample+Seek and launching exact SQL Server queries in the background, letting a visualization show values plus error bars and heatmap uncertainty. On a 70 GB, ~170M-flight dataset approximate results return in ~100 ms on one machine; exact results complete later and caught a single missed flight that the sample missed. Five lab experts and three Microsoft case teams report higher confidence and smoother exploration.
AI-written summary · about
chapters (9)
- 0:00Introduction and motivation
- 2:08Challenges with big data exploration
- 3:58Approximate query processing explained
- 6:28Problems with approximate results
- 7:52Introducing optimistic visualization
- 9:58Demo of Pangloss system
- 12:33User study findings
- 15:05Why optimistic visualization works
- 16:48Future work and closing
related talks
46mExplorations in Interactive Visual Analytics
30mSeeing the needle AND the haystack: single-datapoint selection for billion-poin...
17mAvoiding drill-down fallacies with VisPilot: assisted exploration of data subsets
45mAzure DW and Power BI, reporting at the speed of light!
17mFlight of Insights: Navigating Data Turbulence with SQL Modelling and Visualisation
19mHow to look at your data
30mData @Scale Seattle- Pawel Terlecki
10mThe Future of Data Visualization
