Emerging Challenges in Deep Learning (Simons Institute workshop, 2019)
as the organisers put it
This final workshop will serve two purposes. On one hand, it will survey some of the new, emerging challenges in deep learning: For example, interpretability and verification, failure areas, reinforcement learning, and societal concerns broadly defined (including fairness and privacy). On the other hand, this event will constitute a "capstone” of the program, aiming to support ongoing collaborations between program participants and industry visitors with an intent of sustaining their continuation beyond the program.
who it's for
reinforcement learning researchers and practitioners, deep learning researchers, machine learning theorists, interpretability researchers, fairness and privacy researchers, recommender system engineers, computational biology researchers, robotics and meta-learning researchers
topics
transformer language model reasoning; protein and small-molecule design; human-in-the-loop learning; alignment via reward modelling and decomposition; depth versus width expressivity; MAML meta-learning initialization; hardness of value-function approximation; low-rank transition kernel methods; entropy-regularized off-policy optimization; reinforcement learning for recommender systems; verified deep learning and verification methods; fairness measurement and local explanations
| title | author | length |
|---|---|---|
| Knowledge is embedded in language neural networks but can they reason? | Chris Manning | 50m |
Large transformer language models embed surprising world knowledge but fall short on multi-step reasoning. Scaling (e.g. 117M→1.5B parameters, training on tens… | ||
| Machine Learning-based Design of Proteins and Small Molecules | Jennifer Listgarten | 46m |
Frames protein and small‑molecule design as a practical model-based optimization problem that replaces slow assays with a deep‑net stochastic oracle, then… | ||
| Efficient Deep Learning with Humans in the Loop | Zachary Lipton | 51m |
Argues active learning can cut labels but with caveats: a convolutional NER pipeline runs several× faster and using dropout-based Bayesian uncertainty or… | ||
| Aligning ML objectives with human values | Paul Christiano | 47m |
Frames alignment as a reduction: start with a black‑box optimizer that gets low regret on a model class and aim to compete with that class while the true… | ||
| Is Deeper Better Only When Shallow Is Good? | Shai Shalev-Shwartz | 45m |
Claim: fractal-shaped classification problems can be represented exactly by a ReLU net of depth 2n+1 and width about 5d·r, while any network much shallower… | ||
| Flexible Neural Networks and the Frontiers of Meta-Learning | Chelsea Finn | 50m |
Shows a concrete method — MAML — that learns an initialization so models fine-tune on one to five gradient steps and solve true few-shot problems (5-way 1-shot… | ||
| On the Hardness of Reinforcement Learning With Value-function Approximation | Nan Jiang | 54m |
Asserts that having a realizable value-function class plus exploratory batch data is not enough for sample-efficient RL: a constructed MDP family yields a… | ||
| Reinforcement Learning in Feature Space: Complexity and Regret | Mengdi Wang | 45m |
Shows that tabular sample complexity reaches the known S A / (1−γ)^3 type limits, then moves to low‑rank transition kernel: P(s'|s,a)=φ(s,a)^T M ψ(s') and… | ||
| Optimality and Approximation with Policy Gradient Methods in Markov Decision Processes | Alekh Agarwal | 55m |
Softmax tabular policy gradients with exact gradients asymptotically converge to a global optimum (softmax keeps nonzero action mass), but rates are weak… | ||
| Off-policy Policy Optimization | Dale Schuurmans | 53m |
Argues that naively optimizing importance-weighted expected reward is unstable and high-variance, and instead proposes entropy-regularized surrogate losses and… | ||
| Reinforcement Learning in Recommender Systems: Some Challenges | Craig Boutilier | 52m |
Highlights practical obstacles and partial solutions: recommender action sets are often stochastic action sets that break standard MDPs; user interaction is a… | ||
| Reinforcement Learning via an Optimization Lens | Lihong Li | 47m |
Argues that policy/value learning is a standard optimization problem solved via a primal–dual reformulation that replaces the non‑differentiable max and the… | ||
| Towards Verified Deep Learning | Sanjit A. Seshia | 55m |
Argues that formal methods plus environment models let verification scale: replaces neural nets by an over-approximation (always wrong) and an… | ||
| Integrating Constraints into Deep Learning Architectures with Structured Layers | Zico Kolter | 50m |
Claims that a weight‑tied, input‑injected network can be collapsed to a single implicit layer and trained by solving for its root, implemented as deep… | ||
| The Measure and Mismeasure of Fairness | Sharad C. Goel | 50m |
Argues that common group-based fairness metrics are unreliable: calibration can hide discrimination by withholding distinctions, and equalized false positive… | ||
| Inherent Trade-offs with the Local Explanations Paradigm | Julius Adebayo | 48m |
Compares gradient, SmoothGrad, Integrated Gradients, LIME and influence‑based explanations using concrete checks: reinitializing Inception‑v3 layers (model… | ||
| How to Fail Interpretability Research | Been Kim | 45m |
Field surveys identify 11 recurring ways research stalls and offer concrete checks and datasets to avoid them, for example a synthetic attribution test that… | ||
| Better Learning from the Past: Counterfactual / Batch RL | Emma Brunskill | 41m |
Counterfactual batch RL techniques estimate and optimize policies from logged data using importance-weighted stationary-distribution ratios and doubly robust… | ||
| Designing Robust Learners | Jacob Steinhardt | 53m |
Shows that projecting the corrupted distribution to the closest valid model and outputting that model’s parameter is information-theoretically optimal up to… | ||
| nothing matches | ||