WebRobot: Web Robotic Process Automation using Interactive Programming-by-Demonstration
summary
WebRobot is an interactive programming-by-demonstration tool for web RPA that synthesizes automation programs by speculation-and-validation instead of heavy global rewrite rules. It guesses local loop and selector patterns from a short action trace, validates candidates with trace semantics, and produces stable programs in under one second. Evaluation on dozens of web tasks shows roughly 90% success, users reach intended programs within 10 demonstrations and 90 seconds, and the system synthesizes nested loops (2-level in 32 tasks, >3 levels in 6 tasks).
AI-written summary · about
chapters (12)
- 0:00Start
- 1:49Key idea: interactive programming-by-demonstration
- 2:41Our system vs previous systems
- 3:37Synthesis input
- 4:03Synthesis problem
- 5:07Search based synthesis doesn't work
- 5:32Our idea: leverage action trace
- 5:47Existing rewrite-based synthesis
- 7:19Using existing techniques is not practical
- 12:09Evaluation
- 13:17User study
- 15:53Rules using Egg
related talks
34mRPA Development: What Is Process Robotization and How Far Can I Scale?
25mHarnessing the Power of Automation Testing and RPA
06mRataplan: Resilient Automation of User Interface Actions with Multi-modal Proxies
09mSituated Live Programming for Human-Robot Collaboration
33mAutomating Your Browser and Desktop Apps
24mBots & Humans: Maintaining AI agent and automation in an AI-bot web
47mAutomating Web Workflows with LLMs
24mAutomated Flaw Detection for Industrial Robot RESTful Service