Do You See What I See? Bring Live Pedestrians into an Outdoor Collaborative Mixed Reality Experience
summary
System streams live pedestrian positions from a laptop camera through a server into a virtual replica built with Bentley iTwin Capture and rendered on a Meta Quest 3, aligning AR and VR spaces with QR markers and homography and smoothing tracks with a common filter after YOLO detection. A within-subject study with 21 participants compares no pedestrians, recorded tracks, and real‑time tracks; latency is under one second and spatial error ≈10 cm, and real‑time avatars significantly increase perceived realism, mutual awareness, and sense of visiting together, with most participants treating avatars as representations of actual on‑site people. HoloLens
AI-written summary · about
chapters (7)
- 0:00Introduction to research
- 1:28Mixed reality collaboration concept
- 3:14Addressing information asymmetry
- 4:18System components and setup
- 5:41User study design and conditions
- 8:54Results and conclusions
- 9:16Q&A session
related talks
10mPerceived realism of pedestrian crowds trajectories in VR
27mHoloportation: Virtual 3D Teleportation in Real-time
40mHolographic Telepresence with Hololens
10mMixed Reality check
18mDreamWalker: Substituting Real-World Walking Experiences with a Virtual Reality
51mThere's a Hologram in my UI!
44mBetween Two Realities: Testing AR / MR
23mFuture of User Interaction: Mixed Reality, AI and IoT