Papers.Data.Code
Paper

Context Language Models

Rulin Shao, Shannon Zejiang Shen, Junjie Oscar Yin, Yuetai Li, Minheng Wang, Hamish Ivison, Radha Poovendran, Nathan Lambert

Rather than external memory policies, the LM edits its own context file in place. This makes context management a native behavior that can be prompted or RL-trained.

arXiv ↗Sep 29, 2026Code ↗

  • llm
  • nlp
  • agents
  • context management
  • reinforcement learning
Composite overview with qualitative examples, benchmark comparisons, and headline results showing core claims
Composite overview with qualitative examples, benchmark comparisons, and headline results showing core claims

What it is

Language models edit their own live context as a file instead of relying on external summarizers or fixed memory tools, improving long-horizon agent performance and efficiency. Instead of harness-defined summaries or fixed memory actions, the LM directly rewrites its live context as a file. This gives unrestricted, learnable context control, extends to multi-agent setups, and supports instruction tuning and RL for context strategy.

Key results

  • BrowseComp-Plus: +11.4% accuracy with 21.5% fewer FLOPs vs best baseline
  • 12-hour EdgeBench: +5% score with 59% fewer FLOPs vs summarization
  • RL on Qwen3.5-9B: 28.8% -> 42.5% on BrowseComp-Plus with 12% fewer FLOPs

Making context management intrinsic to the model yields better long-horizon agent behavior than fixed external memory policies, while also lowering compute.

One useful ML email a week.

More from Sep 28 – Oct 04

The whole week →

Picked Oct 2

Picked Oct 1

Worth your time, not all of your time.

We track new ML work across papers, code, and data, then keep only the pieces worth a closer look. Selected, not collected.

Papers.Data.Code

The Weekly ML Signal

Selected, not collected.

One useful ML email a week: the papers, repos and datasets worth your attention.

Worth your time, not all of your time.