What it is
Turns a pretrained text-to-video diffusion model into a single feed-forward, text-instructed vision model for many video perception tasks. It argues video generation is a stronger general-purpose vision pretraining signal than prior SSL. Instead of task-specific vision backbones or image-only pretraining, it repurposes a text-to-video diffusion backbone into a single-pass perception model steered by prompts, using one unified architecture, head, and loss across dense and sparse video tasks.
Key results
- Comparable to D4RT and VGGT-Ω with 7× to 500× less training data
- Often matches or surpasses specialists like DepthAnything3, SAM3, D4RT, VGGT-Ω
- Beats V-JEPA and VideoMAE backbones under comparable settings
Large-scale text-to-video generation learns reusable spatiotemporal and language-aligned features that transfer better than standard video SSL to broad perception tasks.