What it is
Post-trains computer-use models for game control using synthetic RL experience in 16 verifiable environments after SFT on curated gameplay demos. Instead of relying only on demonstrations or game-specific RL, it learns transferable game control from synthetic environments built around shared control demands. It extends CUA actions with relative mouse motion, holds, and composite inputs, then PPO-trains on verifiable rewards.
Key results
- 38.33% on MineDojo-300 for 9B vs 10.54% SFT and 12.63% best baseline
- 41.04% on MineDojo-300 for 35B-A3B vs 23.38% SFT
- MineDojo coverage at >=4/8 successes rises from 20 to 136 tasks for 9B
Shows that synthetic RL environments organized by shared control skills can transfer to held-out real games beyond demonstration-only training.