What it is
Turns temporary context gains into persistent model parameters by distilling context-conditioned behavior into mergeable FFN updates while preserving other capabilities. Instead of relying on retrieval or task-specific editing interfaces, it consolidates context-induced behavior into FFN/MoE parameters. It generates prospective queries from the context, then distills context-conditioned outputs into mergeable low-rank or structural updates, with replay for sequential writes.
Key results
- 50 sequential factual edits on 14B: 88.9% CR and 99.8% RR
- Beats AlphaEdit by +66.2 CR points and +3.8 RR points on that setting
- On 4B-120B dense, MoE, text and multimodal backbones, sits on the CR-RR frontier
Context-conditioned behavior can be internalized as targeted FFN updates with minimal interference to the model's original capabilities.