Effect Handlers for Choice-Based Learning
summary
Combines algebraic effects and handlers with loss continuations to make choice points first-class for optimizing decisions. Shows a Haskell effect-handler library, a semantics and a translation removing loss continuations, and worked examples: binary-search pivot selection, model selection via gradient-descent handlers, simple hyperparameter search, and Q-learning with a mutable Q-table as an effect parameter. A JAX/XLA prototype runs with about a 10% overhead versus a baseline implementation.
AI-written summary · about
chapters (9)
- 0:00Choice-based learning intro
- 1:28Effect handlers for choice-making
- 2:34Effect handlers quick intro
- 6:24Parameters and RL examples
- 7:53Loss continuations for model selection
- 9:18Hyperparameter optimization with handlers
- 9:39Reinforcement learning with handlers
- 11:46Implementation and efficiency
- 12:50Q&A on nested gradient descent
related talks
49mHaskell for choice-based learning
01h 09mLatent Effects and Handlers + Automatic Differentiation via Effects and Handlers
47mIntroduction to Algebraic Effects
17mMachine Learning Primitives as Algebraic Effects
29mAutomatic Differentiation via Effects and Handlers in OCaml
27mEffectful
49mWhat is a (side) effect?
59mA Welcome to Causal Probabilistic Programming