How to test software without writing tests

on YouTube ↗10m

" summary

Claims an automated tester called SQLancer that pairs random query generation (it reuses tools like SQLsmith) with a metamorphic oracle named ternary logic partitioning (TLB). It generates databases and queries, partitions results into true/false/null, and compares counts to spot inconsistencies. The implementation finds many real bugs (reports include over 450 unique issues in major DBMSes and a MySQL fix) and is presented as practical and effective.

AI-written summary · about

" chapters (13)

1. 0:00Intro
2. 0:31Database Management Systems DBMS!
3. 1:37Automatic Testing Core Challenges
4. 2:27Test Oracle
5. 2:47Motivating Example
6. 4:28Ternary Logic Partitioning (TLP)
7. 4:44Scenario: Coffee Kitchen
8. 4:57Tangerines vs. Clementines
9. 6:06Insight
10. 7:17Example: MySQL
11. 8:48Generalizing the Technique
12. 9:09Metamorphic Testing
13. 9:23Summary & Takeaway

" related talks

• 36mSQLancer: Automatically Finding Bugs in Database Management SystemsManuel Rigger · SPLASH 2020 REBASE
• 06mSQLancer: Finding Logic Bugs in Database Management SystemsManuel Rigger · Percona Live Online 2020
• 15mFinding Bugs in Database Systems via Query PartitioningManuel Rigger, Zhendong Su · OOPSLA 2020
• 55mMuch ado about testingNicolas Rinaudo · ScalaIO 2019
• 23mMetamorphic TestingMatthias Hamburg · HUSTEF 2022
• 49mHow do we do it? Testing and Verifying bugs for released softwareLalit Choudhary · TestDriven Conf 2022
• 11mDetecting Transactional Bugs in Database Engines via Graph-Based Oracle ConstructionManuel Rigger, Zhendong Su, Si Liu, Zu-Ming Jiang · OSDI 2023
• 58mTDD for database related code, how is it possible?Mikalai Alimenkou · JavaDay 2013