What it is
An on-policy self-distillation method for diffusion post-training that turns image-level rewards into explicit intermediate denoising targets. Instead of implicit RL credit or endpoint reweighting, it builds bounded positive/negative clean-output targets from reward gradients at sampled denoising states, then fits them as detached supervision and refreshes targets on-policy via EMA.
Key results
- Best held-out score in 19 of 20 matched settings across 2 backbones
- Up to 44.0% better than the strongest competing method
- Cuts training GPU-hours vs DiffusionNFT by 40% on SD3.5-M, 63% on Z-Image-Turbo
Explicit on-policy target construction makes diffusion reward alignment both more efficient and easier to diagnose than implicit reward-driven updates.