What it is
A controlled 120-run study of continued pre-training for game agents, isolating data volume, diversity, and model size across virtual worlds. Instead of scaling game agents with entangled recipe changes, it isolates data, diversity, and model size in CPT and shows a two-phase pattern: early shared interface learning, then world-specific gains. Diversity helps transfer to related games, not distant ones.
Key results
- Scaling law fits: R² 0.995 in-corpus loss and 0.982 Minecraft loss
- At 1,600h, mixed-data S2-S5 average ~2.8× OGA score vs S1
- 27B-S1 drops below its base model on all 3 non-Minecraft benchmarks
Continued pre-training mainly builds transferable control primitives early; sustained generalization needs diverse worlds and still breaks on distant ones.