Programming Languages for Automatic Differentiation: What Now?
summary
Claims that AD is a functor and that backprop is higher-order drive a survey of recent papers, formal correctness proofs, and a lambda-calculus style program transformation that preserves asymptotic cost. Implementations matter: industry tools extract straight‑line programs (TensorFlow/PyTorch), while a direct Python prototype trying to follow the pure functional transform fails for performance and linearity reasons. A key empirical limitation is that conditionals on reals break correctness, so errors are narrow but unavoidable; no clear higher‑order example yet shows practical speed or usability benefits.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 0:30Background
- 8:50Differentiable Programming
- 14:46Backprop
- 17:06What Now
- 21:43Implementation
- 25:19Questions
- 28:16Discussion
related talks
39mDifferentiable Functional Programming
30mAutomatic Differentiation in PCF
14mNominal Semantics for First-class Automatic Differentiation
11mAbstract Interpretation for Automatic Differentiation
59mDifferentiable Programming in C++
29mAutomatic Differentiation via Effects and Handlers in OCaml
29mAutomatic differentiation, a tale of two languages
26mAdding AD to Scheme by Differentiating the Interpreter