What it is
Adds camera sightlines as positional coordinates to a pretrained video DiT, enabling first-frame video generation that follows a specified camera trajectory. Generate videos from a first frame, text prompt, and camera trajectory while preserving the base image-to-video prior by encoding each token with its camera ray.
Key results
- Adds camera control with under 0.1% new parameters
- Keeps pretrained RoPE bit-exact and starts from unchanged Wan2.2 DiT
- Accepts custom OpenCV c2w poses of shape [81,3,4] or [81,4,4]
Shows camera motion can be injected into a video DiT as part of the positional coordinate system, not a separate control branch.