CrystalBox: Efficient Model-Agnostic Explanations for Deep RL Controllers
summary
CrystalBox trains a neural black-box model-agnostic explainer on replayed system traces (eg Puffer) by labeling state+action pairs with horizon-wise, component-wise returns and learning to predict those values. It outputs decomposed reward forecasts (quality, quality-change, stalling) for alternative actions over a bounded horizon, runs for adaptive bitrate and congestion control, requires no policy access, costs only a forward pass at runtime, and shows more accurate future-component estimates than baseline predictors, enabling concrete tradeoff comparisons (higher short-term quality vs risk of long stall).
AI-written summary · about
chapters (12)
- 0:00CrystalBox explainability
- 1:17RL in systems importance
- 2:50RL background example
- 4:39Systems RL properties
- 7:56Public RL traces
- 8:17Explainability for black-box RL
- 9:43Future action explanations
- 11:11Explainer as supervised model
- 12:35CrystalBox advantages
- 13:39User interpretation of outputs
- 14:03Results on ABR and congestion control
- 15:09Summary future research
related talks
14mClosing the Train-Inference Divide for Asynchronous RL at Scale
41mBetter Learning from the Past: Counterfactual / Batch RL
26mStop Explaining Black Box Models, Use Interpretable Models Instead
11mLinear Predictors in Reinforcement Learning: ...
56mA Tutorial on Reinforcement Learning II
48mPractical Model-based Algorithms for Reinforcement Learning and Imitation Learning, with...
34mOffline Reinforcement Learning and Model-Based Optimization
53mAn Integrated Cognitive Architecture