Video Action Models
summary
Argues that pre-training a generative video backbone and adding an action decoder yields much higher sample efficiency for robot manipulation — reported as 10x more sample efficient on teleop fine-tuning compared with VLM-based approaches — and an oracle experiment where giving the true future video produces near-perfect actions. Uses Flux 3 video backbone collaboration and a custom hand and stack, running models locally on an RTX 5090 at about 100 ms inference; data strategy layers include wearable glove data between web video and scarce robot teleop, and infrastructure uses Ray Data streaming to cut time-to-first-batch versus hours, with near-100% success on an industrial kitting task using ~12% of action data.
AI-written summary · about
chapters (12)
- 0:00Introduction and company overview
- 1:53Why video action models
- 4:18Mimic Video model architecture
- 8:51Flux Mimic collaboration and results
- 10:58Real-time performance and deployment
- 13:46Ray Data for scalable training
- 19:22Q&A session part 1
- 21:20Q&A session part 2
- 23:03Q&A session part 3
- 24:12Q&A session part 4
- 26:45Q&A session part 5
- 31:21Closing and summary
related talks
18mRobotics: why now?
24mState of Open Video Generation Models
23mFLUX, Open Research, and the Future of Visual AI
16mKeynote: Robot Foundation Models
16mReal-World Robotics as the Next Frontier for AI?
17mMorphe: High-Fidelity Generative Video Streaming with Vision Foundation Model
26mDeploy Your Own VideoGen Model
14mTowards Unobtrusive Physical AI: Augmenting Everyday Objects with Intelligence and Robotic Moveme...