What it is
Regularizes RAEs by training decoder and DiT on random encoder-layer subsets instead of one fixed fusion, improving robustness and narrowing the reconstruction-generation trade-off. Instead of picking one heuristic or gated layer fusion, train decoder and generator on random normalized subsets of encoder layers. This makes them robust to cross-layer disagreement and supports full, sparse, or single-layer latents without changing the pretrained encoder.
Key results
- Unguided gFID drops 3.01→2.21 by replacing only the decoder
- Joint FuseReg cuts DiT-Base unguided gFID 13.96→9.93
- One decoder gets 27.52 dB PSNR on k=23 and 25.13 dB on single-layer l11
Training downstream RAE components for layer-fusion robustness reduces the mismatch between reconstruction-friendly and generation-friendly latents.