What it is
An online RL method for diffusion models that replaces unstable NFT-style reward regression with aggregated prompt-level guidance plus adaptive restoration. Instead of NFT's forward-process targets and EMA tuning, it aggregates per-prompt rewards into one shared reward-improving direction for reverse-rollout states, then adds a trust-region-masked second update for drift-adaptive restoration.
Key results
- 12.02x fewer GPU hours than DiffusionNFT at matched late-stage reward
- 11.72x fewer GPU hours than Flow-GRPO-Fast at matched late-stage reward
- Mean training reward is 28.8% above DiffusionNFT and 53.0% above Flow-GRPO-Fast
Prompt-level reward aggregation can keep diffusion RL efficient while removing the instability that NFT handled with tuned EMA.