Empowering Children to Create AI-Enabled Augmented Reality Experiences
summary
Capybara is an AR visual programming environment that lets children create and animate 3D characters using speech-to-3D generative models, record movements with on-device auto-rigging and body tracking, and program interactions with the physical world via object-detection sensing blocks. The team builds a block-based AR editor where kids drag-and-drop behaviors, generate accessories, and replay recorded animations; a 20-child study in the US and Argentina shows strong engagement and self-representative avatars but notes limitations in rigging artifacts and a small set of recognizable objects.
AI-written summary · about
chapters (15)
- 0:00Introduction and motivation
- 1:05AI and AR natives today
- 2:29Constructionism in AI AR
- 3:12Introducing Capybara tool
- 3:34Create animate interact process
- 4:15Generating 3D models with AI
- 4:57Animating with body movements
- 6:19Interacting with physical world
- 7:23User study findings
- 8:27Self-representation in creations
- 9:11Everyday life story creation
- 9:54Challenges and AI alignment
- 10:37Pedagogical potential
- 10:59Future directions
- 12:23Q&A and teacher roles
related talks
52mRead, Write, and Tinker with AI
38mAI + AR Superpowers!
18mAI for an Augmented World
06mCan Children Emulate a Robotic Non-Player Character’s Figural Creativity?
10mMeet the Speakers
43mDeveloping AI Agents for Disability and Self Determination: Lessons from Project RAISE
22mAI-Powered Personalized Learning
14mBuilding AI + XR apps faster with XR Blocks