Interleaving Large Language Models for Compiler Testing
summary
A two‑phase framework decouples expensive LLM use: an offline database of 500,000 validated functions generated and profiled by LLMs, and an online composition of function bricks that stitches those snippets into large, valid C programs without calling LLMs. The prototype LegoFuzz finds 66 bugs in GCC and LLVM (about half are miscompilations), achieves ~146× speedup and +50k lines of coverage versus baselines, and uses ~100 iterations as a sweet spot for deep, real compiler bugs.
AI-written summary · about
chapters (13)
- 0:00Lego compiler testing intro
- 0:21LM compiler testing limits
- 1:04Scaling LM-driven fuzzing
- 2:28Lego’s two-stage approach
- 3:34Offline stage: real-world code
- 4:58Validating functions for reuse
- 6:01Online stage: program dependencies
- 7:47Lego’s bug discovery results
- 8:51Ablation study on Lego
- 9:35Comparison with SOTA tools
- 10:15Future work and apps
- 10:57Summary and conclusion
- 11:19Q&A
related talks
53mLocknote: How Badly Do We Want Correct Compilers?
25mEvaluating An LLM's Ability To Code
59mCompiler-assisted Translation Validation
53mSemantics-based program manipulation in the age of LLMs
53mFrom Within: Compiler Testing and Validation via Compilers
16mELFuzz: Efficient Input Generation via LLM-driven Synthesis Over Fuzzer Space
22mBoosting Compiler Testing by Injecting Real-World Code
44mAccelerating Software Development: The LLM (R)evolution