Why RL Environments Are All You Need for Building Agents
summary
Argues that RL environments are the critical ingredient: Bespoke Labs curates sandboxed environments and verifiers, runs SFT and RL and uses prompt evolution like Japa to iterate. Shows Snowflake results across 103 environments comparing GLM 5.2 and Opus 4.7, and a Credit Karma deployment that cuts hallucinations, lowers latency and cost, and scales to 100M+ users. Recommends engineering bespoke environments, building data flywheels that surface failure trajectories for targeted post‑training and harness updates.
AI-written summary · about
chapters (13)
- 0:00How to Scale Test Automation Across Large Engineering Teams
- 4:19Where Test Automation Breaks as Teams Grow
- 7:24Using Self-Healing AI to Improve Test Automation
- 9:56Scaling Playwright Across 14,000 Tests
- 14:31Combining AI Reasoning With Deterministic Test Execution
- 18:04Building an Autonomous AI Test Automation Workflow
- 23:10Improving Test Coverage From Business Requirements
- 26:15Risk-Based Test Selection at Scale
- 29:57How Self-Healing Tests Diagnose and Repair Failures
- 31:31AI-Powered Root Cause Analysis for Test Failures
- 33:37Governance and Safety for AI Test Automation
- 37:12Lessons From Scaling AI Test Automation at Microsoft
- 42:52Balancing Control and Intelligence in Enterprise Test Automation
related talks
19mRL Environments at Scale
27mScaling an Open Environments Ecosystem for Reinforcement Learning
41mLet LLMs Wander: Engineering RL Environments
19mTraining Agentic Reasoners
36mSkyRL: Democratizing RL Training for Researchers
39mLet LLMs Wander: Engineering RL Environments
27mSkyRL: A Modular RL Library for LLM Agents
41mBuilding LLM Agents – Challenges and Learnings