Paper

SPADE: Self-Play in Adaptive Synthetic Executable Environments

A self-play RL setup where one LLM writes executable Gym-style environments and learns to solve them, using hint-based regret to adapt task difficulty online.

arXiv ↗Aug 19, 2026Code ↗

  • nlp
  • llm
  • reinforcement learning
  • self-play
  • tool use
Detailed pipeline showing environment designer and reasoning agent with policy updates
Detailed pipeline showing environment designer and reasoning agent with policy updates

What it is

A single LLM learns both to write executable training environments and solve them, creating an adaptive self-play curriculum for reasoning and tool use. Instead of training on fixed task pools or frozen generators, the model learns an Environment Designer that writes full Gym-style executable environments and a solver that acts in them; designer reward is the solve gap with vs. without privileged hints, pushing tasks to the agent's capability frontier.

Key results

  • +13.9 on ACEBench-Agent over the strongest fixed-env baseline at 30B
  • +5.7 on BFCL v4 multi-turn at 30B over the strongest fixed-env baseline
  • +5.3 avg across 8 held-out math/science/code/reasoning evals at 30B

Training the environment generator itself, rather than keeping the task pool fixed, yields stronger transfer as the agent scales.

One useful ML email a week.

Worth your time, not all of your time.

We track new ML work across papers, code, and data, then keep only the pieces worth a closer look. Selected, not collected.

Papers.Data.Code

The Weekly ML Signal

Selected, not collected.

One useful ML email a week: the papers, repos and datasets worth your attention.

Worth your time, not all of your time.