Bayesian Deep Learning with Edward (and a trick using Dropout)

on YouTube ↗39m

" summary

Claims that Edward (TensorFlow + probabilistic random-variable classes) runs blackbox variational inference on deep nets and that MC Dropout is formally equivalent to approximate Bayesian inference (Gal and Ghahramani). Implements BBVI vs MC Dropout on benchmarks (Boston Housing, Yacht, etc.), finding MC Dropout usually gives lower RMSE for two-layer 50-unit nets, BBVI improves when width drops to 10. MC Dropout estimates use many stochastic forward passes (order 10,000); results suggest mean-field BBVI is too restrictive for wide nets.

AI-written summary · about

" chapters (16)

1. 0:00Bayesian deep learning intro
2. 2:24Standard deep learning drawbacks
3. 4:08Why Bayesian methods matter
4. 5:35Bayesian neural networks
5. 7:19Exact inference challenges
6. 9:04Probabilistic programming overview
7. 11:33Edward extends TensorFlow
8. 14:25Simpler probabilistic modeling
9. 16:12Variational inference in BNNs
10. 18:59Dropout as Bayesian inference
11. 23:57BBVI vs MC Dropout
12. 27:31Experimental results analysis
13. 30:16Closing remarks
14. 30:39Q&A part one
15. 34:10Q&A part two
16. 37:02Q&A part three

" related talks

• 36mBayesian Deep learning with 10% of the weightsRob Romijnders · PyData Amsterdam 2018
• 57mTutorial on Deep Learning IIIRuslan Salakhutdinov · Foundations of Machine Learning Boot Camp 2017
• 46mKnow What You Don't Know: Tools to Understand Uncertainty in DLJulian Eisenschlos · PyData Córdoba Argentina 2019
• 01h 22mDeep Probabilistic Methods with PyTorchChris Ormandy · PyData London 2018
• 37mVariational Inference in PythonAustin Rochford · PyData DC 2016
• 35mUncertainty in deep learningMadhu Gopinathan · Anthill Inside 2018
• 38mActive Learning for 3D Mesh Semantic SegmentationErik Harutyunyan · PyData Yerevan 2022
• 38mBlack-Box Variational Inference for Probabilistic ProgramsHongseok Yang · Uncertainty in Computation 2016