Realtime Video AI with Diffusion Models
summary
Claims realtime means generating one second of video in under one second and delivering usable frames with only 2–4 denoising steps; latency tests in demo show interactive performance even across continents using StreamDiffusion v2 for video-to-video and LongLive for text-to-video served via Daydream and explored in Scope. Techniques actually used: KV caching, latent propagation, optical flow warping and temporal attention. Demo exports reproducible results via saved prompts and seeds and runs on consumer GPUs (4090+), with single-step output much faster but visibly lower quality.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 1:08Real-time video AI defined
- 3:40Real-time vs traditional video AI
- 6:09Diffusion models for video
- 8:18Challenges in real-time diffusion
- 11:30Daydream Scope tool
- 12:54Demo: Stream Diffusion v2
- 14:40Demo: Long-Life text-to-video
- 16:56Video diffusion breakthroughs
- 18:22Try real-time video AI
- 19:25Q&A
related talks
47mBest Practices for Real-time Intelligent Video Analytics
45mDiffusion to Prod: vLLM-Omni Unifies Multimodal Inference
50mAI'll Be Back: Exploring Diffusion Models for Image, Audio & Video
10mDiffusion models
41m8K Codec Low-Latency Livestreaming, and AI Streaming
21mReal-Time Video Super Resolution for Live Streaming with WebGPU
27mOn challenges and considerations for real time AI processing
19mReal-Time AI: Low Latency Solutions for Interactive Applications