Combining Shape-Changing Interfaces and Spatial Augmented Reality Enables Extended Object Appearance
summary
Combining shape-changing mechanics with projection-based spatial augmented reality extends an object’s appearance by delegating coarse form to actuators and fine detail or fast motion to projection. A 6-servo, NinjaFlex-topped tablet with OptiTrack, Arduino Bluetooth, and projection mapping runs a mechanical distance-field algorithm that voxelizes actuator states for real-time fitting to arbitrary 3D targets. Demos include a haptic labyrinth, map navigation using view-dependent shape change to avoid cropping, and an ambient weather display. The prototype runs in real time and the hardware and code are open-sourced.
AI-written summary · about
chapters (15)
- 0:00Introduction and motivation
- 0:30Combining shape-changing interfaces with SAR
- 1:14Limitations of shape-changing interfaces
- 1:36Advantages of spatial augmented reality
- 2:18Complementary benefits of the combination
- 3:26Three core concepts in the framework
- 4:32Animated texture maps and shadow maps
- 5:36Environment maps and perceived transparency
- 6:19Benefits of the combined approach
- 6:40How shape-changing interfaces benefit SAR
- 7:46Implementation and hardware details
- 8:52Mechanical distance field algorithm
- 11:05Three showcase applications
- 12:14Conclusion and open-source resources
- 13:18Q&A on display integration
related talks
09mAdvanced Augmented Reality interactions
19mRetroShape: Leveraging Rear-Surface Shape Displays for 2.5D Interaction on Smartwatches
43mBeyond The Printed Page
07mProjection-based Augmented Reality games
20mFoveAR: Combining an Optically See-Through Near-Eye Display with Projector-Based ...
07mHapticBots: Distributed Encountered-type Haptics for VR with Multiple Shape-changing Mobile Robots
20mSymbiosisSketch: Combining 2D & 3D Sketching for Designing Detailed 3D Objects in Situ
42mSpatial UX & Spatial AI [Keynote]