Beyond Decoding: Meta-Generation Algorithms for Large Language Models
summary
Defines a practical framework called meta-generation that composes token-level decoders with higher-level strategies: chain-of-thought prompting to spend more test-time compute, AlphaCode generating millions of programs then filtering, and four families—chaining, parallel sampling with weighted-majority voting, tree search, and refinement. Shows weighted voting recovers a marginalized correctness limit, REBASE tree search cuts error ~5% vs parallel sampling at equal compute, and composes verifiers/tools (compilers, interpreters) to fix code successfully.
AI-written summary · about
chapters (11)
- 0:00Meta-generation algorithms intro
- 1:24Inference-time compute importance
- 12:24Single-sequence generation
- 24:34Beyond single-sequence generation
- 30:01Chaining as strategy
- 33:22Parallel generation & voting
- 43:09Tree search for generation
- 50:08Refinement & self-correction
- 1:08:10Compute-optimal inference
- 1:12:07Future directions
- 1:14:47Q&A
related talks
39mUnlocking reasoning in Large language models
15mKeynote: Pre-training and Fine-tuning of Code Generation Models
45mHow Generative AI Works: LLMs and RAG Under the Hood
27mUnlocking the Potential of Large Language Models
53mGenerative AI: Overview and use cases
21mType-Constrained Code Generation with Language Models
43mBeyond the Hype: A Realistic Look at Large Language Models
54mLost and Found in Translation