Brute Force Correctness
summary
Describes applying generative differential testing to GraphQL: a schema/query generator (built with Kotest Property examples) runs millions of fuzzed objects through a new Viaduct executor and a control implementation to surface spec-edge bugs. The system finds many failures quickly, drives planner-to-executor architectural changes, and after exhaustive testing the rewritten Viaduct ships and runs nine months with zero GraphQL spec conformance bugs. Results scale to the GraphQL conformance project comparing 16 implementations (13 show failures).
AI-written summary · about
chapters (11)
- 0:00GraphQL edge case quiz
- 4:37The case for fuzz testing
- 6:23Generative testing history
- 7:05Designing a GraphQL fuzzer
- 8:19Implementing the generator
- 11:35Analyzing complex queries
- 12:04Validating production systems
- 18:54GraphQL conformance project
- 20:08Patterns in conformance gaps
- 21:59Conclusion and resources
- 22:37Q&A session
related talks
19mFunctional Testing of GraphQL APIs: Simple Checks That Catch Real Bugs
52mBasics of GraphQL and Testing GraphQL APIs
24mA GraphQL Survival Kit
26mThe missing piece of testing
31mStop Guessing A11y: Auto-Generate Playwright Tests from Your GraphQL Schema
13mEfficient Semantic Comparison of GraphQL Queries
38mPerformance Testing sucks! How to get started testing RESTful APIs and GraphQL
32mTesting GraphQL APIs