Repo

JoyAI-Video-Edit

Edits video streams frame by frame from text instructions, without full-video access or fixed length. Supports live camera and uploaded video inputs.

GitHub ↗Aug 4, 2026Paper ↗

  • multimodal
  • video-editing
  • streaming-inference
  • diffusion

What it is

Real-time instruction-guided video editor for live or uploaded streams that edits causally as frames arrive. The released deployment hits 30.19 FPS at 720x1280. Edit live camera feeds or uploaded videos from natural-language instructions in a causal streaming setup, handling open-ended sequences without waiting for the full clip.

Key results

  • 30.19 FPS end-to-end at 720x1280 in deployment benchmark
  • Processes frames causally without revisiting future frames
  • Supports subject, local, background, style, motion, and reference edits

Shows that open-ended diffusion-based video editing can run interactively in a causal streaming setting.

One useful ML email a week.

Worth your time, not all of your time.

We track new ML work across papers, code, and data, then keep only the pieces worth a closer look. Selected, not collected.

Papers.Data.Code

The Weekly ML Signal

Selected, not collected.

One useful ML email a week: the papers, repos and datasets worth your attention.

Worth your time, not all of your time.