What it is
Turns past discovery trees into replay simulators to improve exploration policies offline, then redeploys them online in a recursive self-improvement loop. Instead of fixed exploration or costly online meta-optimization, it replays recorded discovery trees as simulators to score and revise exploration policies off-policy, while keeping the underlying coding agent unchanged.
Key results
- Lasso: up to 162x fewer agent calls than SimpleTES
- Math optimization: over 50x budget savings vs SimpleTES within 1k gens
- KernelBench: up to 2.09x better kernel performance at equal budget
Shows that logged discovery history can act as a cheap simulator for improving long-horizon exploration policies without repeated online rollouts.