Exploring The Forward Forward Algorithm
summary
Claims a biologically plausible learning rule that replaces the backward pass with two forward passes (online/wake and offline/sleep), uses a per-layer goodness per layer objective (sum of squared ReLU activities) and requires normalized inputs before multiplication. Implementation with Keras/TensorFlow reaches ~90.7% on MNIST versus a baseline 97.5% backprop, trains faster on a blood‑cell task (353–392s vs 1296s) but generalizes worse (test error ~0.53–0.58 vs loss 0.054), and supports pipelined online learning without storing activations.
AI-written summary · about
chapters (11)
- 0:00Start
- 0:47Training A Neural Network
- 3:34Does the brain do backpropagation?
- 5:02Problems
- 9:05What is the forward forward algorithm?
- 10:02Layer Level Cost Function
- 14:37What is real/positive data?
- 15:31What is fake/negative data?
- 16:59Normalizing Inputs
- 22:23Calculating derivatives
- 31:11Blood Cell Dataset
related talks
43m(Deep) Learn You a Neural Net for Great Good!
27mUnderstanding Deep Neural Networks
56mA Gentle Introduction to Neural Networks and making your own with Python
01h 37mTen Steps to Keras
41mLearning TensorFlow
41mReally Deep Neural Networks with PyTorch
01h 25mBuilding & Optimization of Neural Networks from Scratch with Numpy
01h 27mHands-On Computer Vision with PyTorch