Local Agentic Theory for Accessible Mobile Games
summary
Architecture puts tiny reinforcement-learning and reasoning agents directly on phones, trading 200 ms cloud latency vs ~3 ms LiteRT to run perceive‑predict‑decide‑act loops at ~60 Hz, letting sample‑efficient agents adapt difficulty per player. Demo rebuilds Space Invaders with command‑style agents, uses gaze estimation and multi‑model ensembles, and treats crosswords as constraint‑minimisation graphs. Engineering focuses on negotiating the space, time and energy budgets (model weights, state history, frame jank, battery).
AI-written summary · about
chapters (25)
- 0:00Local Agentic Theory for Accessible Mobile Games
- 1:35Speaker Intro - Shafik Quoraishee, Staff Game Engineer, The New York Times
- 3:09Disclaimer: NYT Puzzles Are Human-Made; This Is Exploratory Work
- 5:14AI Has Always Been in Games - From Turing to Pac-Man
- 6:16Finite State Machines as the First Named Game AI
- 7:19Why On-Device Models Matter: Speed, Privacy, Offline, Personalisation
- 10:27AlphaGo to EfficientZero: RL Agents That Solved Games
- 12:28Reinforcement Learning vs Agentic AI - The Real Distinction
- 15:04Space Invaders Rebuilt With an Agentic Solver
- 16:35The Perceive, Predict, Decide, Act Loop
- 17:37On-Device Constraints: The Space Budget for Model Weights and State
- 19:40The Frame Budget - and Why Jank Kills a Game
- 20:41The Energy Budget and Why Devices Aren't Optimised for AI Workloads
- 21:43Negotiating Space, Time and Energy as One Engineering Problem
- 23:13Gaze Estimation and Vision Models as a New Reactivity Layer
- 24:44ARC-AGI-3: Models Still Haven't Reached Human Action-Space Intelligence
- 25:15Crossword Solving as a Constraint-Minimisation Graph Problem
- 26:46Generalist Agents, World Models and Neural Game Generation
- 28:49Accessibility: Active Testing Agents vs Static Checks
- 29:51Focus Traps and Dynamic UI That Static Testing Can't Predict
- 30:52What On-Device Agentic Development Still Needs
- 31:55Q&A: Simulating Accessibility Conditions in a Sandbox
- 35:32Q&A: Do Agents Make Games Accessible by Testing or by Adapting Play?
- 37:04Q&A: Running Inference Every Frame vs on Meaningful State Change
- 39:39Q&A: Reasoning Reliably Under Non-Determinism on Device
