LLM Query Benchmarks: Cypher vs SQL
summary
A realistic enterprise schema (Contexture) and 21 benchmark questions across five complexity levels test multiple LLMs (GPT‑4, Gemini, Claude, Mistral, fine‑tuned Llama variant). Initial runs produce many wrong directions, deprecated Cypher syntax and failed recursive SQL queries — only three models solve one recursive SQL case. Changing relation phrasing and adding few‑shot fixes fixes most Cypher errors, and asking an LLM to produce a verbose SQL schema plus an automatic SQL view for recursion yields a big jump in correctness.
AI-written summary · about
chapters (7)
related talks
27mAI, SQL, and GraphQL Walk into a Fertility Clinic
40mJudgement Day: Benchmarking "Black Box" LLMs With Open Legal Datasets
44mProduction-Grade LLMs & AI Agents
29mAI, SQL, and GraphQL Walk into a Fertility Clinic… LLM-based Medical feature development
40mLLMs & Data Warehouses: Reliable AI Agents for Business (Keynote)
14mText2Cypher Beyond English: Initial Evaluation with Foundational LLMs
40mTechniques to avoid common LLM pitfalls
30mLLMs: Beyond the Hype - A Practical Journey to Scale