Gemino: Practical and Robust Neural Compression for Video Conferencing
Talkat 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI '24) · 2024-04-16..04-18·
summary
System reconstructs 1024x1024 video by sending tiny downsampled frames plus a single high-resolution reference and performing high-frequency-conditional super-resolution that warps reference features to each target pose. Implementation runs atop aiortc and is optimized with a multi-scale architecture and per-person personalization; real-time decoding hits ~30 fps on a Titan X GPU. Paper reports 2.2–5x lower bitrate than VP9/VP8 for comparable perceptual quality and sensible video at ~75 kilobits per second.
AI-written summary · about
chapters (12)
- 0:00Neural video compression intro
- 1:18Traditional codec limits
- 3:02Bandwidth challenges
- 4:28Gemino’s approach
- 7:56High-frequency super resolution
- 9:42Prototype and resolution
- 11:09Model optimizations
- 12:34Gemino vs traditional codecs
- 14:24Gemino vs learned methods
- 16:08Speed and real-time
- 16:50Adapting to networks
- 17:11Conclusion & Q&A
related talks
17mMorphe: High-Fidelity Generative Video Streaming with Vision Foundation Model
17mYuZu: Neural-Enhanced Volumetric Video Streaming
22mAI Enhanced GPU Video Coding: Achieving Joint High Compression Efficiency and Throughput
25mOptimal Video Compression Using Pixel Shift Tracking
33mArchitecting Scalable Multi-Modal Video Search
11mTxt2Vid: Ultra-Low Bitrate Compression of Talking-Head Videos via Text
21mReal-Time Video Super Resolution for Live Streaming with WebGPU
22mMovie Gen Video: State-of-the-art Video Generation