Sensible Agent: A Framework for Unobtrusive Interaction with Proactive AR Agents
summary
Sensible Agent adapts both what to suggest and how to present it using egocentric cameras, Yamnet audio, vision-language models and few-shot chain-of-thought prompts; an expert workshop (n=12) and annotation study (n=40) drive the policy choices. A WebXR prototype built on XRBlocks and an Android Moohan headset enables visual panels, TTS, head and hand gestures; a user study (n=10) shows significant reductions in cognitive load and perceived effort versus a voice-only baseline while maintaining comparable usability and higher preference for the system.
AI-written summary · about
chapters (9)
- 0:00Introduction to Sensible Agent
- 1:28Research question and motivation
- 3:14Framework design and modules
- 4:16User study and findings
- 5:43Prototype implementation
- 7:26System demonstration
- 7:48User study results
- 9:10Discussions and future work
- 9:52Conclusion and Q&A
related talks
11mEnabling richer experiences through multimodal interactions
22mAccessibility and Interaction in a World Beyond Screens
23mVirtual Human Agent for Smart City
01h 01mKeynote Panel: Future of AI agents
55mDon’t Make Me Prompt: Intelligent Interfaces Users Actually Like
19mBeyond agents: AI as a creative partner
28mAgentic AI
29mIs XR the Next Big Thing? Immersion, Collaboration, and More!