Smart Crop and Smart Preview via Video Understanding
summary
Automatic reframing converts 16:9 videos to 1:1 or 4:5 inside Creator Studio, processing thousands of videos and centering the main subject using a saliency map CNN (RGB input) plus text detection, zoom heuristics and temporal smoothing to avoid jitter. Preview generation scores every two seconds with a frozen multimodal classification trunk and a Siamese ranking loss, uses scrub and edit logs plus A/B tests for active learning, and outputs long or short previews via a reusable, cacheable pipeline.
AI-written summary · about
chapters (10)
- 0:00In-stream video understanding
- 2:59Smart crop with saliency maps
- 9:52Smart crop demo
- 10:14Video highlights for previews
- 13:12Training highlights model
- 19:59Preview generation
- 22:31Scaling highlights platform
- 23:54Future directions
- 24:30Q&A
- 37:05Future of video understanding
related talks
27mSummarizing & analyzing videos
20mBuilding a Professional Video Editor on the Cloud, Powered by Machine Learning
33mArchitecting Scalable Multi-Modal Video Search
22mMovie Gen Video: State-of-the-art Video Generation
14mLeveraging Artificial Intelligence for Scene Detection in Broadcast Video
35mLarge-Scale Video Intelligence
36mApps That See 👀 Practical AI Vision for Developers
33mSee More, Be More