What it is
Adds a prospective self-distillation target for RL-trained agents: use completed trajectories to teach what the policy should have anticipated before acting. Instead of learning only from reward contrast or hindsight-to-action distillation, SRD distills post-hoc trajectory knowledge into pre-interaction foresight targets, so even reward-uniform failures or successes provide training signal.
Key results
- Up to +24.2 points over RLVR/self-distillation baselines on 10 tasks
- At 2B, +SRD gets 60.6% success vs 0.0% for RLVR with same budget
- 4B GRPO+SRD gains +10.03 Math avg and +8.33 Code avg over GRPO
Shows that post-hoc trajectories can supervise predictive representations before interaction, recovering learning signal when group-relative RL is blind.