The Enigma of LLMs: on Creativity, Compositionality, Pluralism, and Paradoxes
Talkat Special Year on Large Language Models and Transformers, Part 1 Boot Camp · 2024-09-04..09-06·
summary
Follow-up experiments claim that LLM behavior surprises: a creativity index matches generated text against large web corpora and finds ChatGPT often reproduces internet phrasing more than professional authors do, and RLHF measurably reduces novelty; an InfiniGram index runs over ~3 trillion tokens with sub-100ms n‑gram lookup and <$1k CPU indexing; and a training recipe called implicit chain-of-thought converts supervised CoT into an internalized procedure that reaches ~99%+ accuracy on long multiplications that vanilla training fails to learn.
AI-written summary · about
chapters (9)
- 0:00Introduction and context
- 2:47Creativity thought experiment
- 9:51Measuring linguistic creativity
- 17:23Creativity index methodologies
- 23:05Infini-gram and empirical results
- 27:51Infini-gram technical deep dive
- 44:40Compositionality and math limits
- 52:23Implicit Chain-of-Thought
- 1:05:37Paradoxes and concluding thoughts
related talks
44mUnderstanding the abilities of AI systems: Memorization, generalization, and points in between
37mExpanding the Capabilities of LLMs Through Community Collaboration
24mHarnessing the Power of LLMs in Advanced Analytics
27mExploring the Mechanisms of LLM Reasoning Capabilities
19mLLMs reasoning and agentic capabilities over time
21mThe Coherence Trap: Why LLMs Feel Smart (But Aren’t Thinking)
25mWhy LLMs Still Fail: Reasoning, Robustness, and Safety
07mLightning Talk: What LLMs Won't Ever be Able to Do