Why the float did it NaN?
summary
Explains why floating point representation produces NaN, Inf, and silent underflow/overflow by showing memory layout and examples where subtraction of near-equals loses all precision, multiplication of many probabilities underflows to zero, and naive matrix inversion destabilizes solutions. Demonstrates the LogAddExp trick (log-sum-exp) to add tiny probabilities, recommends avoiding numpy.linalg.inv and using LU decomposition and scipy.linalg.solve instead, and gives concrete failure counts and code-level remedies.
AI-written summary · about
chapters (12)
- 0:00Intro
- 1:50What is a float
- 3:47What are approximations
- 5:23Underflow
- 9:36How often
- 10:31Log
- 11:29Log Probability
- 13:26Logarix
- 15:21Log probabilities
- 16:26Linear equations
- 19:26Solving linear equations
- 24:36Wrap up
related talks
48mDemystifying Floating Point
11mExploiting Verified Neural Networks via Floating Point Numerical Error
41mUnveiling the simple math that powers AI
10mGotchas of Pandas
01h 27mMaking computation easier with cool Numpy tricks
01h 00m“CNL: A Compositional Numeric Library”
35mSecure Numerical Computing is Hard: Lessons from 10 Years of Open Data Science & the Long Road Ahead
50mElevating Precision in C++ - A Journey Below the Surface of Floating-Point