ValScope: Value-Semantics-Aware Metamorphic Testing for Detecting Logical Bugs in DBMSs
Talkat 20th USENIX Symposium on Operating Systems Design and Implementation (OSDI '26) · 2026-07-13..07-15·
summary
ValScope implements a unified SQL query approximation model and an approximation propagation analysis to reason how local mutations (aggregation and expression mutators like M(C1)→MIN(C1) or C1→C1-2) change final results, not just result sets. It generates, mutates and verifies queries, finds 67 unique logical bugs across 6 widely used DBMSs (57 confirmed, 48 predating 2020, one bug present for 20 years), and exposes value-level errors such as incorrect aggregation and numeric computation that set-based MTs miss.
AI-written summary · about
chapters (5)
- 0:00ValScope introduction
- 2:24DBMS testing limitations
- 8:45Value-semantics model
- 11:12Generate mutate verify workflow
- 15:48Evaluation and bug detection
related talks
14mDetecting Logic Bugs in Database Engines via Equivalent Expression Transformation
19mPinolo: Detecting Logical Bugs in Database Management Systems with Approximate...
55mMuch ado about testing
15mDDLumos: Understanding and Detecting Atomic DDL Bugs in DBMSs
06mSQLancer: Finding Logic Bugs in Database Management Systems
36mSQLancer: Automatically Finding Bugs in Database Management Systems
10mHow to test software without writing tests
14mChecking Observational Correctness of Database Systems