Embedded Deep Learning Super Resolution on GStreamer using ONNX Inference Runtime
summary
Demonstration of an edge-ready super-resolution pipeline that integrates a small, APQ-optimized model into GStreamer via an ONNX Runtime backend and a new GStreamer element. The system estimates image-specific blur kernels with an internal-GAN plus cycle consistency in an unsupervised, zero-shot way, yields PSNR/SSIM that match or exceed prior methods, and on a VIM3 NPU the model is significantly smaller than EDSR and runs >3× faster at 2× and ~2× faster at 4× upscaling.
AI-written summary · about
chapters (12)
- 0:00Intro
- 0:44Super-Resolution
- 2:15Why (up)scaling matters
- 3:57Solved Problem
- 5:42What is Super Resolution?
- 7:52Blind Super Resolution?
- 9:24Super Resolution types
- 13:00Kernel Estimation
- 16:25Overview
- 18:42Qualitative Results
- 19:51Quantitative Results
- 22:02GStreamer Implementation
related talks
35mNNStreamer: Efficient and Agile On-Device AI Application Development with Neural-Net Pipelines
24mSuper Resolution: The scaler of tomorrow, here today!
41mIntroduction to Deep Learning Models in Unreal
17mYour Player is Smarter Than You Think: Self-Learning in Adaptive Streaming
21mImage Super-Resolution in JS
17mYuZu: Neural-Enhanced Volumetric Video Streaming
36mApplied AI on the edge
21mReal-Time Video Super Resolution for Live Streaming with WebGPU