Computer-Use Agents: From SFT to Classic RL
summary
Describes concrete computer-use agent systems and datasets, including OpenAI and Anthropic Claude computer-use APIs, Code Agent beta, and the Android Wild dataset (~30k tasks) where a 1.5B model reaches about 71% offline and ~74% online. Presents training recipes showing supervised fine-tuning (SFT) then Direct Preference Optimization (DPO) gives modest gains, while reward-models plus PPO variants (JPO/GRPO) add further improvement; reports leaderboard results (OpenAI agent ~42.9% success on OSWorld, top models in the low 40s), explains evaluation environments (MacOS VMs and screenshot-only agents) and demos that commercial agents are functional but remain slow and brittle.
AI-written summary · about
chapters (19)
- 0:00Computer agents intro
- 1:11What is a computer agent
- 3:00Agent architecture pipeline
- 5:50Agent training environments
- 7:37Commercial agent solutions
- 9:46Agent demo
- 13:01SFT to RL training
- 16:32Training dataset creation
- 17:55Existing agents overview
- 20:48Qwen 2.5 model
- 22:23Reinforcement learning for agents
- 26:17DPO method benefits
- 29:32GRPO algorithm
- 32:05UIСA training with GRPO
- 33:32Classic RL unused reasons
- 34:15Android Wild research
- 37:28Agent evaluation benchmarks
- 38:57Metrics and performance
- 40:21OS World leaderboard
related talks
49mAI Agents: Collaboration in Real-Life Applications
40mAI Agents in Practice
19mTraining Agentic Reasoners
21mDemocratizing AI with Open Source Multi-Agent Systems
44mOpen-Source and Science in the Era of Foundation Models
01h 01mAI Agents In-Depth – Function Calling, MCP and Tool Use Under the Hood
07h 47mAI Agents Conference 2026 - Day 5
27mHow to Build AI Agents and Keep Your Sanity