Neural Networks are Programs, Too

on YouTube ↗37m

" summary

Claims neural networks are programs so functional techniques (Finally Tagless, Haskell types) yield automatic differentiation by dual numbers, a DSL that compiles to a DAG with hash consing, and algebraic vector spaces to generalize gradients. Implements forward and reverse modes, an interpreter for simply typed lambda calculus, and shows a compositional NN encoding (XOR, loss) but stops short of GPU optimization; performance remains unoptimized. Watch if wanting a concrete DSL+AST+AD implementation rather than high‑level motivation.

AI-written summary · about

" chapters (21)

1. 0:00Introduction
2. 1:05Deep Learning Framework
3. 1:47Newer Networks
4. 2:14Gradient Descent
5. 2:49Recurrent Neural Network
6. 3:25Convolutional Neural Network
7. 3:56Train Neural Network
8. 9:09Simple Selling Strategies
9. 9:54Example
10. 14:38Prototype
11. 16:18Space
12. 18:02Gradient
13. 19:43Exponential
14. 20:25Basis
15. 23:33Direct
16. 27:13Automatic differentiation
17. 29:39New network
18. 30:20Topology
19. 30:52High Order Functions
20. 31:22Standard Term
21. 33:47NonStandard Term

" related talks

• 34mDifferentiable Functional ProgrammingNoel Welsh · Scala Days Berlin 2018
• 36mProgramming Languages for Automatic Differentiation: What Now?Damiano Mazza · LAFI 2022
• 44mFast Neural Networks… a no brainer!Riccardo Terrell · Lambda Days 2017
• 39mDifferentiable Functional ProgrammingNoel Welsh · Scala Days New York 2018
• 59mWhats and Whys of Neural Network Verification (A Declarative Programming Perspec...Ekaterina Komendantskaya · PADL 2024
• 37mMonadic Deep LearningYang Bo, Xiaolei Wang · Strange Loop 2017
• 46mCode + AI: Will Robots Take Our Coding Jobs? ML Applied to ProgrammingStephen Magill · GOTO Chicago 2019
• 33mLearn Neural Networks With Go - Not Math!Ellen Körbes · DevDays Europe 2019