Neural Networks Demystified: Part 7
summary
A neural net maps hours slept and hours studied to predicted test score and initially fits the training data so tightly it produces implausible dips where more studying lowers score. Overfitting is diagnosed with a train/test split and plots of training vs testing error; model complexity is tied to nine weights so the rule of thumb is 10x rule (≈90 examples). Regularization uses an L2 penalty (sum of squared weights) with a tunable Lambda hyperparameter to shrink weights and bring training and test errors closer; supporting IPython notebooks are on Welch Labs.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 0:24Data
- 1:14Signal and Noise
- 3:27Training and Testing
- 3:58Fixing Overfitting
- 4:17Regularization
- 5:09Conclusion
related talks
46mOn the Foundations of Deep Learning: SGD, Overparametrization, and Generalization
01h 02mUnexpected Test Losses from Generalization Theory?
31mWait, You Test Neural Networks Too?
33mMachine Learning Under Test
01h 37mTutorial: Implicit Bias II
58mBenign, Tempered, or Catastrophic: A Taxonomy of Overfitting
40mNeural Networks And Spurious Correlations
52mNeural Networks Walkthrough