Squiggle: Multimodal Lasso Selection in the Real World
summary
Squiggle lets users sweep an invisible lasso with a pinch ring and smart glasses gaze to produce image masks (Project Ares hardware; VR data collected on Meta Quest Pro). A VR study (12 participants, 65 targets) shows drawing angular size correlates with target size and gaze often traces lasso edges; a real-world study (12 users, 23 targets) fine-tunes a prompt encoder and mask decoder on Efficient SAM and finds lasso and hybrid mode significantly improve mask accuracy for compound objects and groups over gaze+pinch alone.
AI-written summary · about
chapters (6)
- 0:00Introduction
- 1:09Squiggle project overview
- 3:56Virtual reality data collection
- 7:02Real-world prototype
- 9:49User study results
- 11:11Conclusion and Q&A
related talks
22mPinpointing: Precise Head- and Eye-Based Target Selection for Augmented Reality
18mMasque: Exploring Lateral Skin Stretch Feedback on the Face with Head-Mounted Displays
09mTangibleData: Interactive Data Visualization with Mid-Air Haptics
17mLearning Cooperative Personalized Policies from Gaze Data
12mRemote Collaboration With Mixed Reality Displays: How Shared Virtual Landmarks Facilitate ...
09mThe Subconscious Director: Dynamically Personalizing Videos Using Gaze Data
10mNon-isomorphic Interaction Techniques for Controlling Avatar Facial Expressions in VR
19mProjective Windows: Bringing Windows in Space to the Fingertip