Automatic differentiation, a tale of two languages
summary
Compares Python frameworks (TensorFlow Jax PyTorch) with a dozen or so AD backends in Julia and claims choice order flips: Python picks framework first, Julia picks the scientific library first then an AD backend. Demonstrates dual numbers and Heron iteration to show operator overloading limits, explains forward vs reverse cost (gradient ≈ one eval via reverse), highlights tooling built like differentiationinterface.jl and low‑level projects such as enzyme (LLVM/XLA) and notes small bus factors for many Julia libraries.
AI-written summary · about
chapters (14)
- 0:00Welcome
- 1:52Automatic differentiation matters
- 4:21Symbolic vs numeric limits
- 7:49Algorithmic differentiation
- 9:35AD systems internals
- 13:28Python vs Julia AD
- 15:52Python vs Julia differences
- 18:42Future of differentiable languages
- 19:45Unified Julia AD interfaces
- 20:52AD beyond simple functions
- 21:33Q&A
- 23:23Julia AD backends
- 25:30Framework vs backend
- 28:14Cross-language AD integration
related talks
36mProgramming Languages for Automatic Differentiation: What Now?
43mJulia: A Fresh Approach to Machine Learning
54mGeneralized Physics-Informed Learning through Language-Wide Differentiable Programming
30mProductivity and performance in one programming language - A tour of Julia
01h 07mJulia Tutorial
39mDifferentiable Functional Programming
29mAutomatic Differentiation via Effects and Handlers in OCaml
01h 27mIntroduction to Julia for Scientific Computing and Data Science