Response to “Synthesizing Input Grammars”: A Replication Study
summary
Replication reimplementation uses a different codebase and evaluation pipeline and reports F1 scores lower than paper but still shows Glade outperforming L- and RPNI baselines on the same benchmarks; original artifacts are public artifacts available and reproduce the paper’s numbers. The replication finds Glade fails on palindrome language* (a context‑free case outside typical program inputs), tends to emit large or overly permissive grammars for XML/grep, and shows results sensitive to seed input generation and sampling choices.
AI-written summary · about