What it is
Defines a unique token-level credit for LLM RL and turns it into PACT, an actor-then-critic update that aligns critic targets to the updated policy. Instead of treating token credit as heuristic, it proves a unique credit under three axioms and uses that to redesign actor-critic training: update actor first, then train the critic with importance-corrected targets aligned to the new policy.
Key results
- 72.87% avg math accuracy, beating GRPO by 8.80 pts and PPO by 13.16
- 67.4% SWE-bench Verified pass@1, +2.0 over GRPO and +2.4 over PPO
- 83.12 on AIME 2025, above GRPO's 76.50
Shows that critic-policy misalignment is a major bottleneck in LLM RL, and fixing it materially improves long-horizon reasoning and coding.