Programming Large Language Models with Algebraic Effect Handlers and the Selection Monad
summary
Pangolin models LLM calls as algebraic effects and uses a selection monad to search and score nondeterministic outputs; handlers expose a multi‑shot continuation so code can sample an LM multiple times (example uses 16 samples) and run a loss continuation to pick the best result. The authors provide a sketch language and a tiny WIP implementation that demonstrates best‑of, assertion‑based retries, and custom scoring, but a full open‑source system and formal semantics remain future work.
AI-written summary · about
chapters (10)
- 0:00Algebraic effects for LLM programming
- 1:03Limitations of prompting
- 2:08Pangolin with effect handlers
- 3:54Naive handler for LLM interaction
- 5:59Sampling outputs with handlers
- 6:43Selection monads for loss/search
- 7:43Best-of pattern with choose/score
- 9:30Assertions and retries
- 10:55Future work and Pangolin’s vision
- 11:59Q&A
related talks
26mLanguage Model Agents Enable Semi-Formal Programming
53mSemantics-based program manipulation in the age of LLMs
46mBeyond Basic Prompting: Supercharging Open Source LLMs with LMQL's Structured Generation
51mTowards a Pragmatic Understanding of Large Language Models
18mReductive Analysis with Compiler-Guided Large Language Models for Input-Centric Code(…)
40mSurviving the Agentic Hype with Small Language Models
49mA Practical Approach to Using LLMs: Peculiarities and Pitfalls
43mLarge Language Models for Devs: from zero to your first LLM application