Composable Effect Handling for Programming LLM-integrated Scripts
summary
Claim: treating LLM calls, I/O and concurrency as abstract operations discharged by handlers separates workflow from effects and yields real gains. Prototype implements a restrictive Python with-based effect handlers that compose handler stacks for development, deployment and testing. Built examples include synchronous, asynchronous and replay handlers that preserve sequential logging order while parallelizing LLM requests, producing measurable speedups (a reported 10× speedup on a Tree-of-Thoughts case study).
AI-written summary · about
chapters (6)
- 0:00LLM software challenges
- 2:55LLM software features
- 4:39Effects in agent workflows
- 7:13Logic versus effectful ops
- 9:22Composing handlers
- 15:04Q&A
related talks
01h 00mFrom LLM Theory to Practical Agentic Implementations
01h 16mHow Will LLMs Change Haskell, Programming and Open Source?
53mSemantics-based program manipulation in the age of LLMs
36mComposable Architecture for AI Systems: APIs, MCP, Skills, Sub-Agents, and Wor…
33mDesigning LLM Applications for Massive Datasets
30mEfficient programming with Scala and LLMs
10mRethinking collaboration: LLMs, teams, and cognitive load
26mContext Engineering: Practical Techniques for Improving Agent Quality Today