Content Aware Encoding for low latency live streaming encoders using deep learning
summary
A deep learning model predicts per-segment target bitrate from lookahead statistics and luma histograms and runs on encoder inference hardware with minimal overhead. The pipeline is a fully connected regression network ingesting SAD, motion vector and activity histograms and four-frame lookahead, trained so the ground truth bitrate yields VMAF ≥ 90. Results show ~35% bitrate savings with <1 VMAF loss on low complexity clips and ~9% savings on medium complexity, negligible savings on high complexity.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 1:09Limitations of static bitrate ladders
- 2:51Encoder factors driving quality
- 4:41Problem and deep learning approach
- 6:49Input features and neural network
- 8:36Training data and ground truth
- 9:39Model training and evaluation
- 11:07Integration with encoder
- 12:53Experimental setup and hardware
- 13:57Results and bitrate savings
- 15:22Live demo and quality comparison
- 16:28Conclusion and Q&A
related talks
12mMeasuring live video quality with minimal complexity, now available for everyone!
21mWhat to do after per-title encoding
16mMAE: More Adaptive Video Encoder for Consistent Low Latency in High-Quality Real-Time...
30mMLDS 2019: Anubhav Shrivastava of Voot at Machine Learning Developers Summit 2019
17mLiFteR: Unleash Learned Codecs in Video Streaming with Loose Frame Referencing
29mHow to build a high quality 'live' HD VP9 Encoder
17mYour Player is Smarter Than You Think: Self-Learning in Adaptive Streaming
22mAI Enhanced GPU Video Coding: Achieving Joint High Compression Efficiency and Throughput