Joint IFML/MPG Symposium 2024
as the organisers put it
This workshop brings together researchers from the NSF AI Institute for Foundations of Machine Learning (IFML) and the Simons Institute fall programs on Modern Paradigms in Generalization, and on LLMs and Transformers. The workshop will focus on relationships between learning algorithms, generalization, and advanced architectures in LLMs. With this goal, the workshop will host speakers from these areas and foster collaborations cross-cutting these themes.
who it's for
theoretical machine learning researchers, statistical machine learning researchers, research scientists working on LLMs, optimization and algorithm designers, graduate students in ML and CS, researchers in robust and unsupervised learning
topics
method of moments for linear dynamical systems; Ho-Kalman system identification; subsampled smooth calibration error; overlap-gap property in combinatorial problems; transductive priors for unbounded losses; Chebyshev scalarization in multi-task learning; InfoNCE and contrastive learning analysis; self-training filter-and-retrain; omnipredictors for single-index models; meta-generation algorithms for large language models; prompt-tuning with trainable input tokens; score matching reductions for Gaussian mixtures
| title | author | length |
|---|---|---|
| Learning from Dynamics | Ankur Moitra | 43m |
Given one long input-output trajectory, the paper proves polynomial-time parameter recovery for linear dynamical systems under marginal stability plus bounded… | ||
| The Truth About Your Lying Calibrated Forecaster: How to Design Truthful Calibration Measures | Nika Haghtalab | 35m |
Shows that standard calibration measures like ECE are gameable: a manipulative forecaster gets ECE≈0 while a truthful forecaster pays Θ(√T) variance or even… | ||
| Some Easy Optimization Problems Have the Overlap-Gap Property | Tselil Schramm | 38m |
Shows that some easy problems like S–T shortest path in Gn,Θ(log n)/n nevertheless satisfy an ensemble OGP: correlated graph sequences force near-optimal paths… | ||
| Bypassing the Impossibility of Online Learning Thresholds: Unbounded Losses and Transductive Priors | Nikita Zhivotovskiy | 42m |
Shows that knowing the set of covariates in advance and using transductive priors with continuous exponential weights yields regret bounds for unbounded losses… | ||
| Revisiting Scalarization in Multi-Task Learning | Han Zhao | 44m |
Claims that linear scalarization cannot in general trace the Pareto front for linear multi‑task models, proving concretely that when hidden width q >= k the… | ||
| Understanding Contrastive Learning and Self-training | Sujay Sanghavi | 45m |
Analyzes InfoNCE on linear nets trained on a Gaussian mixture with an explicit augmentation model (pairs drawn from same component with probability delta). In… | ||
| Omnipredicting Single-Index Models with Multi-Index Models | Kevin Tian | 40m |
This work constructs omnipredictors for single-index models by a sharp, agnostic analysis of the Isotron algorithm, giving a learner that is eps-competitive… | ||
| Beyond Decoding: Meta-Generation Algorithms for Large Language Models | Sean Welleck | 01h 22m |
Defines a practical framework called meta-generation that composes token-level decoders with higher-level strategies: chain-of-thought prompting to spend more… | ||
| Robust Mixture Learning when Outliers Overwhelm Small Groups | Fanny Yang | 42m |
Claims an algorithm that attains order-optimal error for each mixture mean while keeping the extra output size minimal, provably requiring list size about k +… | ||
| Learning General Gaussian Mixtures With Efficient Score Matching | Vasilis Kontonis | 35m |
Provides an algorithm that constructs a sampling oracle within epsilon total-variation for a k-component Gaussian mixture under well-conditioned covariances… | ||
| Talk by Mahdi Soltanolkotabi (University of Southern California) | Mahdi Soltanolkotabi | 34m |
Adds trainable input tokens to a frozen foundation model (prompt-tuning) and analyzes a single‑head attention proxy on a Gaussian mixture style data model… | ||
| Testing Noise Assumptions of Learning Algorithms | Arsen Vasilyan | 35m |
Claims an efficient procedure that tests noise assumptions and either outputs a classifier with a certificate of optimality or rejects; extends RV23 testable… | ||
| Talk by Ali Kavis (UT Austin) | Ali Kavis | 27m |
Presents a parameter-free second-order optimistic method for monotone convex–concave min–max problems that removes no line search or subsolvers, proves the… | ||
| nothing matches | ||