Flexibility General Average for ML
summary
Presents a continuous, differentiable operator called Generalized Average (GA) that interpolates min, max, arithmetic, geometric and harmonic means; shows two experiments: using GA for hyperparameter tuning to bias toward fewer false positives (fraud-detection style) and replacing global pooling in GNNs where GA initialized near max matches or slightly exceeds sum/max pooling across ~100 trials. Implementation is in an open-source Python package, training uses TensorFlow + Spectral, and reported challenges include numerical stability when jointly optimizing the p parameter in Graph Attention Networks.
AI-written summary · about
chapters (14)
- 0:00Flexibility generalized average intro
- 1:27Presenters and collaborators
- 2:29What is an average
- 4:35Continuum of averages
- 6:21General average and power mean
- 8:11Weighted averages properties
- 10:43Visualizing generalized average
- 13:54Generalized F1 and F beta scores
- 17:04Graph neural networks recap
- 21:01Global pooling in GNNs
- 23:07Generalized average in GNNs
- 27:43Research challenges and next steps
- 28:23Generalized average applications
- 29:28Q&A
related talks
01h 01mML for discrete optimization: Theoretical foundations
01h 20mThe elusive generalization: classical bounds to double descent to grokking
01h 10mModern paradigms of generalization, the heliocentric model of Aristarchus,...
46mSmooth Nonconvex Min-Max Optimization
42mWhat (Doesn't) Make Learning Algorithms Generalize?
49mGeneralized Energy-Based Models
47mIs Minimax Optimization the Right Framework to Understand Learning in Minimax Games?
01h 01mOptimization's Untold Gift to Learning: Implicit Regularization