Je ne regrette rien: Teaching Machine Learning Models Regret Avoidance
summary
Claims that integrating regret-sensitive loss functions into existing models lets them account for downstream components, implemented by mapping game-tree regret nodes to training sample weights. Practical recipes include XGBoost sample_weight, TensorFlow sample_weight, and PyTorch cross_entropy with reduction='none' multiplied by regret weights. A tunable regret factor is required and is grid-searched. Deployed in a payments product called revenue accelerate this change yields about 300,000 more transactions per month (~€20M) while keeping model complexity constant.
AI-written summary · about
chapters (19)
- 0:00Introduction
- 1:07Complex ML needs global optimization
- 3:18Regret in decision-making theory
- 5:04Components of ML systems
- 6:52Renters approval use case
- 9:21Sequentiality and uncertainty in decisions
- 11:50System as a game tree
- 14:41Regret nodes in decision tree
- 16:46Regret in ML models
- 19:20Regret via sample weights
- 22:34Adyen Revenue Accelerate case
- 24:41Results and business impact
- 25:45When to apply this approach
- 26:48Q&A
- 27:09Tuning regret parameter
- 28:53Continuous targets and regret
- 30:59Avoiding cyclic regret issues
- 33:09Dynamic regret weighting
- 34:36Closing remarks
related talks
01h 17mOnline Reinforcement Learning and Regret
56mA Tutorial on Reinforcement Learning II
31mRegret Minimization for Stochastic Shortest Paths
36mModel-Based Reinforcement Learning with Value-Targeted Regression
46mDynamic Regret Minimization for Bandits without Prior Knowledge
42mBypassing the Impossibility of Online Learning Thresholds: Unbounded Losses and Transductive Priors
49mFast Rates for No-Regret Learning in General Games
01h 02mThe Key Ingredients of Optimizing Test-Time Compute and What's Still Missing