Repo

DreamX-Creator

Creates synchronized audio-video from a first frame and prompt, then upscales video to 2K in one refinement step while keeping audio unchanged.

GitHub ↗Sep 1, 2026Paper ↗

  • multimodal
  • audio-video-generation
  • video-super-resolution
  • 2k
Composite showing audio-driven video generation outputs with 2K quality and diverse scenes
Composite showing audio-driven video generation outputs with 2K quality and diverse scenes

What it is

Framework for native joint audio-video generation from a first frame and prompt, plus a 1-step refiner that upgrades outputs to 2K video. Generate synchronized video with audio from a first frame and text prompt, then refine the result or any input video to 2K with a separate 1-step super-resolution model.

Key results

  • Ships a 7B joint audio-video generator with released weights
  • Includes an autoregressive 1-step 2K refiner based on SR-DiT 5B
  • Supports single-GPU inference and multi-GPU sequence-parallel runs

Shows that open native joint audio-video generation can be packaged with a practical 2K refinement stage.

One useful ML email a week.

Worth your time, not all of your time.

We track new ML work across papers, code, and data, then keep only the pieces worth a closer look. Selected, not collected.

Papers.Data.Code

The Weekly ML Signal

Selected, not collected.

One useful ML email a week: the papers, repos and datasets worth your attention.

Worth your time, not all of your time.