What it is
Builds a team of persistent AI research agents with different scientific backgrounds, private memory, and adaptive scheduling to autonomously discover better linear attention designs. Instead of a single agent or uniform multi-agent search, it uses discipline-specialized researchers with private histories, shared evidence, and UCB-style allocation to drive orthogonal exploration and iterative refinement of architectures.
Key results
- At 1.3B, CFDN outperforms GDN2 on LM, CSR, synthetic retrieval, and RWR
- Search evaluated 63 candidates within an 80-iteration research budget
- Removing backgrounds, private history, or adaptive allocation gave weaker discoveries
Autonomous discovery improves when agents differ in expertise, retain their own research trajectories, and are allocated experiments adaptively.