ProMemAssist: Exploring Timely Proactive Assistance Through Working Memory Modeling in Multi-Moda...
summary
A smart-glasses system models a user’s working memory in real time from camera and microphone signals, encoding perception as 7-slot perception buffer items and binding them into a 4-slot episodic buffer via CLIP embeddings and an LLM; a timing predictor optimizes value versus cost (displacement and phonological interference) to decide deliver/delay/discard. A built prototype runs on sensor streams, and a user study with 12 participants shows more selective assistance and higher engagement than an LLM-only baseline.
AI-written summary · about
chapters (8)
- 0:00Introduction and motivation
- 1:29Problem and system goals
- 2:10Working memory modeling pipeline
- 4:16Proactive assistance timing strategy
- 6:21Live system demonstration
- 8:57User study and results
- 10:22Lessons and future work
- 11:04Conclusion and Q&A
related talks
15mMemory-Augmented AI Agents: Enhancing Long-Term Reasoning and Context Awareness
23mParallel Eyes: Exploring Human Capability and Behaviors with Paralleled First Person View Sharing
28mAgentic Memory
22mAI-Powered Personalized Learning
12mInteractive Planning-based Cognitive Assistance on the Edge
06mPrompto: Investigating Receptivity to Prompts Based on Cognitive Load from Memory Training ...
27mMemory Engineering: Going Beyond Context Engineering
46mThe Anatomy of Memory in Humans and AI Agents