Holographic Video Calling

on YouTube ↗17m

" summary

A practical holographic call pipeline captures RGB+depth, generates point clouds, compresses them and streams over real networks; experiments show VPCC 200× compression but still about 14 Mbps for 700k vertices, depth is far more sensitive to quantization than RGB (H.264 at CRF2 produces visible holes and floating artifacts), and the reference encoder needs 30 s/frame encoding on an i7. Adaptive bitrate must split budget across RGB/depth/views, WebRTC's Google congestion control underutilizes available bandwidth, and stronger FEC/ concealment plus new depth codecs and quality metrics are required.

AI-written summary · about

" chapters (10)

1. 0:00Holographic video calling intro
2. 0:49Ideal vs practical setup
3. 2:13Functional components
4. 3:58Data compression challenges
5. 6:51Point cloud compression
6. 8:17VPCC performance
7. 10:26Adaptive bitrate for 3D
8. 12:14Transport and latency
9. 14:42Reducing latency & bandwidth
10. 16:25Summary & future research

" related talks

• 12mHolographic video callingNitin Garg · Demuxed 2021
• 27mHoloportation: Virtual 3D Teleportation in Real-timeAdarsh Kowdle, Pushmeet Kohli, Vivek Pradeep, David Kim, Sarah Mennicken, Sean Fanello, Shahram Izadi, Cem Keskin, Christoph Rhemann, Sergio Orts-Escolano, Wayne Chang, Yury Degtyarev, Philip L Davidson, Sameh Khamis, Mingsong Dou, Vladimir Tankovich, Charles Loop, Qin Cai, Philip A Chou, Julien Valentin, Shenlong Wang, Sing Bing Kang, Yuliya Lutchyn · UIST 2016
• 31mDeveloping VR Media TechnologyAndrew Scherkus · Demuxed 2016
• 32mWebRTC UpdateNIKLAS BLUM · Kranky Geek WebRTC Brazil 2016
• 40mHolographic Telepresence with HololensM. Förtsch, T. Endres · Codemotion Amsterdam 2018
• 21mReal-Time Video Super Resolution for Live Streaming with WebGPUJohn Bartos, James Hurley · Demuxed 2024
• 28mAn Introduction to WebVRMichaela Lehr · JSConf AU 2016
• 18mGaze-Driven Video Delivery: Science Fiction or Viable Link to the Metaverse?Yuriy Reznik · Video @Scale Fall 2022