Paper

Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent

Scales horizon rather than model size by training on long tool-grounded trajectories, then distills six domain teachers into one 35B agent.

arXiv ↗Jun 29, 2026Code ↗

  • nlp
  • llm
  • agents
  • knowledge distillation
  • tool use

What it is

A 35B MoE agent model matches or beats some 1T models by scaling long-horizon trajectories and multi-domain agent skills instead of parameter count. Instead of scaling parameters, it scales agent horizon: builds a knowledge-action graph for 45K-token trajectories, then merges six domain specialists into one 35B student via domain-routed on-policy distillation with salient vocab alignment.

Key results

  • SEAL-0: 56.4, above Kimi-K2.6 55.0 and DeepSeek-V4-pro 42.3
  • IFBench: 80.6, above Kimi-K2.6 73.5 and DeepSeek-V4-pro 75.9
  • FrontierScience-Olympiad: 79.0, above Kimi-K2.6 76.0 and DeepSeek-V4-pro 78.0

Long-horizon, tool-grounded training plus routed multi-teacher distillation can let a 35B agent reach 1T-class performance on agent benchmarks.

One useful ML email a week.

Worth your time, not all of your time.

We track new ML work across papers, code, and data, then keep only the pieces worth a closer look. Selected, not collected.

Papers.Data.Code

The Weekly ML Signal

Selected, not collected.

One useful ML email a week: the papers, repos and datasets worth your attention.

Worth your time, not all of your time.