Lightning Talk: What LLMs Won't Ever be Able to Do
summary
Claims that LLMs fail at cryptic crosswords, backed by a live test: ChatGPT is given clues, produces a wrong answer, accepts correction but could not justify answer, and its explanation is nonsensical. Tools used are a commercial LLM and hand-crafted cryptic clues including a recently released C++‑themed cryptic crossword (QR code), plus a human solver who finishes under ten minutes. Technical claim: LLM training favours surface patterns so it fails where language is intentionally subverted.
AI-written summary · about
related talks
25mWhy LLMs Still Fail: Reasoning, Robustness, and Safety
01h 18mThe Enigma of LLMs: on Creativity, Compositionality, Pluralism, and Paradoxes
27mExploring the Mechanisms of LLM Reasoning Capabilities
21mThe Coherence Trap: Why LLMs Feel Smart (But Aren’t Thinking)
31mBeyond LLMs and Maturing RAG use cases
47mHarnessing LLMs for Next-Level Security Analysis
30mLLM Query Benchmarks: Cypher vs SQL
35mEvaluating LLM systems: the nuts and bolts