Gaze-Driven Video Delivery: Science Fiction or Viable Link to the Metaverse?

on YouTube ↗18m

" summary

Gaze-driven delivery claims up to 500x pixel reduction for displays and even up to 8000x reduction for VR by encoding a 1° foveal patch instead of full frames, but derives a concrete delay–savings equation: 33 ms → ~3° ROI, 66 ms → ~5° ROI, 500 ms → ~31° (no gain). Practical system uses device-to-gateway last-link with tile-based streaming plus a low-res full-frame base layer, requires ~30–60 ms round‑trip, frame rates ≫60 fps, no buffering or retransmits, and therefore only works locally rather than across long-haul networks.

AI-written summary · about

" chapters (10)

1. 0:00Gaze-driven video delivery intro
2. 1:12Gerald's 1988 vision
3. 1:55Human vision principles
4. 5:01Pixel reduction potential
5. 7:08Latency impact on gaze tracking
6. 10:19Delay thresholds for viability
7. 12:29Low-latency system constraints
8. 13:54Pragmatic system architecture
9. 16:39Metaverse potential
10. 17:47Summary

" related talks

• 09mHow to Throw Out 95% of Pixels in VR, Without Anyone Noticing!!Amrita Mazumdar · !!Con West 2019
• 17mYuZu: Neural-Enhanced Volumetric Video StreamingAnlan Zhang, Chendong Wang, Bo Han, Feng Qian · 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI '22)
• 27mHoloportation: Virtual 3D Teleportation in Real-timeAdarsh Kowdle, Pushmeet Kohli, Vivek Pradeep, David Kim, Sarah Mennicken, Sean Fanello, Shahram Izadi, Cem Keskin, Christoph Rhemann, Sergio Orts-Escolano, Wayne Chang, Yury Degtyarev, Philip L Davidson, Sameh Khamis, Mingsong Dou, Vladimir Tankovich, Charles Loop, Qin Cai, Philip A Chou, Julien Valentin, Shenlong Wang, Sing Bing Kang, Yuliya Lutchyn · UIST 2016
• 17mPath to Metaverse-ready NetworksShah Mijanur Rahman · Open Networking & Edge Executive Forum 2022
• 17mHolographic Video CallingNitin Garg · RTC @Scale 2022
• 17mMorphe: High-Fidelity Generative Video Streaming with Vision Foundation ModelTianyi Gong, Zijian Cao, Zixing Zhang, Jiangkai Wu, Xinggong Zhang, Shuguang Cui, Fangxin Wang · NSDI '26
• 25mFirefly: Untethered Multi-user VR for Commodity Mobile DevicesChendong Wang, Feng Qian, Xing Liu, Christina Vlachou, Kyu-Han Kim · USENIX ATC '20
• 46mScience based Design Decision Making for Virtual RealityAz Balabanian · Øredev 2018