LiFteR: Unleash Learned Codecs in Video Streaming with Loose Frame Referencing
Talkat 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI '24) · 2024-04-16..04-18·
summary
LiFteR replaces tight, sequential frame references with LFR (loose frame referencing) and a binary-tree referencing schedule, letting a learned DNN codec decode many frames in parallel; implementation adds an LFR dispatcher, an LFR-aware learned codec using self‑attention, and a virtual buffer for AB to hide undecoded segments. Evaluation shows learned codecs on GPU normally decode ~30 fps vs >120 fps for CPU traditional codecs, and LiFteR achieves up to 3.2× frame‑rate and up to 23.8% QoE gain vs the best learned baseline (19.7% vs traditional).
AI-written summary · about
chapters (14)
- 0:00Video streaming challenges
- 1:20Video codec evolution
- 2:25Learned vs traditional codecs
- 3:51Learned codec pipeline
- 5:17Loose frame referencing
- 6:23Frame scheduling problem
- 8:54Binary tree for references
- 10:19Codec design with binary tree
- 11:45Self-attention for frames
- 12:07VoD streaming challenges
- 13:33ABR with virtual buffer
- 14:17Performance analysis
- 16:06Lessons and future work
- 16:47Summary & Q&A
related talks
16mSwift: Adaptive Video Streaming with Layered Neural Codecs
17mContent Aware Encoding for low latency live streaming encoders using deep learning
21mWhat to do after per-title encoding
17mYour Player is Smarter Than You Think: Self-Learning in Adaptive Streaming
21mLearning in situ: a randomized experiment in video streaming
20mBoosting Video Codec With RISC-V Vector Extension
30mMLDS 2019: Anubhav Shrivastava of Voot at Machine Learning Developers Summit 2019
41m8K Codec Low-Latency Livestreaming, and AI Streaming