SQLancer: Automatically Finding Bugs in Database Management Systems
summary
SQLancer is an automated tester (Java/Maven, GitHub) that generates small databases and thousands of SQL queries and validates results with three concrete oracles: pivot row oracle (checks a single guaranteed row, finds ~100 bugs), non‑optimizing rewrite (derives an unoptimized/query rewrite to expose optimizer bugs, ~160 bugs), and tuple/aggregate partitioning (partition + compose to test aggregates, ~200 bugs). It reports over 450 distinct DBMS bugs across SQLite, MySQL/MariaDB, PostgreSQL, CockroachDB and others, and many were fixed rapidly.
AI-written summary · about
chapters (24)
- 0:00SQLancer introduction
- 1:28DBMS testing need
- 2:30Logic bugs in DBMSs
- 4:19Differential testing limits
- 6:02SQLancer overview
- 7:28Test case generation
- 8:11Pivot query synthesis
- 10:39PQS validation
- 12:50PQS results
- 13:51Advantages and disadvantages of PQS
- 14:54NORC introduction
- 15:38Optimization bug example
- 17:24NORC query rewrites
- 20:58NORC results vs PQS
- 22:51Advantages and limitations of NORC
- 23:31Ternary logic partitioning
- 25:17TLP strategies
- 26:45TLP aggregate testing
- 29:54TLP CockroachDB bug
- 31:22TLP effectiveness
- 32:26Advantages and disadvantages of TLP
- 32:46SQLancer strategy
- 33:29SQLancer adoption
- 34:33Summary
related talks
06mSQLancer: Finding Logic Bugs in Database Management Systems
10mHow to test software without writing tests
15mFinding Bugs in Database Systems via Query Partitioning
11mDetecting Transactional Bugs in Database Engines via Graph-Based Oracle Construction
15mDDLumos: Understanding and Detecting Atomic DDL Bugs in DBMSs
14mDetecting Logic Bugs in Database Engines via Equivalent Expression Transformation
18mApparently, you can "debug" your SQL queries
19mPinolo: Detecting Logical Bugs in Database Management Systems with Approximate...