Lost in High-dimensional Space? Python to the Rescue!

on YouTube ↗20m

" summary

Claims that sparse grid combinations cut the exponential blowup of full d‑dimensional grids (example: level 6 corresponds to 2^6=64 -> 65 points per axis in the full grid) and that growth becomes much weaker up to 20 dimensions with thousands of points per axis. Implementation uses Python high‑level control plus numpy (indexing objects, where, unraveling), itertools and simple linear combinations, produces the figures shown, and becomes the basis of the Python Grids and Fields toolkit under development.

AI-written summary · about

" chapters (16)

1. 0:00Introduction
2. 2:00Lost in Highdimensional Space
3. 8:10Scale Lattice
4. 11:06Intuitive Picture
5. 12:19HighLevel Complexity
6. 13:50LowLevel Complexity
7. 14:21Gridded Field Representation
8. 14:57Virtual Linearization
9. 15:29Numpy
10. 15:50Slice Intrinsic
11. 16:24Classic Combination
12. 16:50Finding Neighbors
13. 17:32Compact Stencil
14. 17:59Python Grids and Fields
15. 18:48MapReduce Framework
16. 19:29Conclusion

" related talks

• 01h 34mAlgorithmic High-Dimensional Geometry IAlex Andoni · Big Data Boot Camp 2013
• 51mTwo Concise Representations for High-Dimensional Visual DataJohn Wright · Workshop on Succinct Data Representations and Applications (Simons Institute Big Data 2013)
• 01h 04mHigher-dimensional Expansion of Random Geometric ComplexesTselil Schramm · Structural Results 2023
• 36mLow-rank matrix approximations in PythonChristian Thurau · PyData Berlin 2014
• 01h 34mAlgorithmic High-Dimensional Geometry IIAlex Andoni · Big Data Boot Camp 2013
• 44mScalable Algorithmic Primitives for Data ScienceRichard Peng · Randomized Numerical Linear Algebra and Applications 2018
• 01h 03mRecent Progress in High-Dimensional LearningAnkur Moitra · Probability, Geometry, and Computation in High Dimensions Boot Camp (Simons Institute, Fall 2020)
• 42mDimensionality Reduction Via Sparse MatricesJelani Nelson · Workshop on Succinct Data Representations and Applications (Simons Institute Big Data 2013)