Video @Scale 2024
who it's for
video engineers, machine learning engineers, platform engineers, research scientists, infrastructure engineers, mobile engineers
topics
text-to-video foundation models (MovieGen); flow matching training for video; latent space compression for generation; video transcoding pipeline pre-filtering; Cosmos-based microservices encoding; on-device deep learning upscaling; SAM 2 promptable video segmentation; efficient segmentation models (EfficientSAM, SqueezeSAM); generated video evaluation and benchmarks; speech-driven captions and lip sync pipelines; multilingual dubbing and translation inference; zero buffer rate and viewer experience metrics
| title | author | length |
|---|---|---|
| Live Fireside Chat | Abhinav Kapoor | 37m |
Covers concrete progress: Luma AI’s Dream Machine launches late June 2024, hits about 25 million users and cuts generation for a 10‑second clip from ~200… | ||
| Movie Gen Video: State-of-the-art Video Generation | Ishan Misra | 22m |
Movie Gen is a text‑to‑video foundation model that generates HD videos up to 16 seconds by training a TA encoder plus a Transformer on hundreds of millions of… | ||
| How Captions Enables Anyone To Tell Their Story With Video | Corbyn Salisbury | 19m |
Captions claims to let anyone make shareable talking videos by generating speech-driven lip sync from text using an internal multi‑stage Transformer called… | ||
| Meta's AI Translations Inference | Sravan Rekula, Jordi Cenzano, Amisha Jaiswal | 20m |
Pipeline performs autodub and lipsync at Meta scale using the Seamless model (six languages) plus 10+ auxiliary models for language ID, VAD, ASR, noise… | ||
| Building Netflix's New Video Pipeline | Liwei Guo | 18m |
Netflix rebuilds its video pipeline on Cosmos-based microservices with a clear three-layer contract API → workflow → compute; encoding becomes a distinct… | ||
| On-device Video Playback Upsampling | Wen Li | 18m |
Mobile playback uses Luma-only DL upscaling plus classical filters to cut latency and compute: models undergo pruning and 8‑bit quantization and leverage… | ||
| Live Q&A: Day 1 | Diana Chen | 29m |
Live Q&A covers concrete systems and numbers: a video generator runs at 24 fps native generation (vs prior 4 fps) and ships 1,000+ non‑cherry‑picked samples on… | ||
| Video @Scale 2024 - Day 1 | 03h 29m | |
Panel and talks name concrete wins and engineering tradeoffs: generative video systems report real usage and speedups (Dream Machine: 25M users, 19.2s per 10s… | ||
| SAM 2: Segment Anything in Images & Video | Chay Ryali | 22m |
Segment Anything Model 2 is a transformer with streaming memory that performs promptable segmentation on images and videos, trained on a new SAV dataset with… | ||
| Challenges in Generated Video Evaluation | Deepti Ghadiyaram | 19m |
Recent generative video evaluation relies heavily on CLIP-based scores for frame and prompt alignment but CLIP often misses detailed prompt elements and poorly… | ||
| Efficient Segment Anything | Bilge Soran, Yunyang Xiong | 17m |
EfficientSAM and SqueezeSAM are compact reworks of Segment Anything that keep quality while cutting compute: EfficientSAM uses mask image pretraining and a… | ||
| Metrics Driven Obsession with Viewer Experience | Khawaja Shams | 17m |
Argues that the single best end-user metric is Zero Buffer Rate (ZBR) and shows how obsessing on it exposed a hidden failure mode when building a realtime… | ||
| Using Performance Signals to Improve Video Recommendations | Zhi Long Tan, Wen Zhang | 14m |
Client devices send bandwidth estimation and real‑time playback signals (starts, stalls, wait time, memory/CPU hints) to ranking models that downrank risky… | ||
| Rethinking the Video Transcoding Pipeline: How Pre-Filtering Holds the Key to Efficiency and Quality | Ramzi Khsib | 19m |
Proposes a production-ready input pre-filter that combines pixel- and transform-domain methods with a perceptual thresholding module to preserve texture and… | ||
| Video @Scale 2024 - Day 2 | 02h 47m | |
Unified segmentation model Sam 2 extends image-only SAM with streaming memory, object pointers and occlusion scores, and claims 6X faster image inference, 3x… | ||
| nothing matches | ||