What it is
Adapts PPO clipping to rollout staleness in asynchronous RL, tightening only high-mismatch updates to stabilize training under policy lag and engine mismatch. Instead of PPO's fixed clip for all tokens, SAT shrinks only the outward clip boundary on high-staleness samples, using detached log-ratio as a staleness proxy; ordinary and pull-back updates stay unchanged.
Key results
- 35.83 AIME24 avg@8 at lag 1 with SAT-GSPO w/ R3
- 34.79 AIME24 avg@8 at lag 8 with SAT-GSPO w/ R3
- 34.17 AIME24 avg@8 at lag 1 with SAT-GSPO
Shows that matching trust-region strength to per-sample staleness can stabilize asynchronous RL without globally shrinking all updates.