Context Language Models: 11% Better Answers on 21% Less Compute
Every agent builder eventually writes the same piece of ugly code: a threshold. Context crosses 80% of the window, the harness summarizes the oldest turns, and the model keeps working from a digest it never asked for. A paper published September 29 by researchers at the University of Washington and Meta Superintelligence Labs hands the editor role to the model itself. They call it Context Language Models, and the numbers make my threshold-based defaults feel old.
What a CLM actually is
The core move: context stops being an append-only transcript and becomes a file. The model edits that file the same way it would edit code, with ordinary Bash commands. It deletes a stale search result, rewrites its own plan, compresses twenty tool calls into two lines, or keeps a scoreboard of open questions at the top and updates it in place. Edits sync automatically back to the serving loop, so generation continues on the edited version rather than the original transcript.
Nothing here is a new architecture. The transformer underneath is unchanged. The authors ran the method on models we already know, including Qwen3.6-27B, Qwen3.5-9B, Claude 4.6 Sonnet and GPT-5.6-Sol, and released no trained weights. "Context Language Model" names a capability plus a training recipe, closer to how "tool-using model" describes a permission than how "Mamba" describes an architecture. The harness, RL modules and serving patch are on GitHub under facebookresearch:
# the model edits its own context file with ordinary shell commands
sed -i '118,240d' context.md # drop stale tool results
# the edited file becomes the next promptThe numbers, and the caveats attached
The paper reports CLMs beating every context-management baseline it tested, usually while spending less compute: 11.4% higher accuracy with 21.5% fewer FLOPs on BrowseComp-Plus, 5% higher scores with 59% fewer FLOPs on a 12-hour EdgeBench run, and 65% greater improvement at equal compute on a 24-hour multi-repository agent-swarm task. The baselines include Codex-style summarization, MEM1, Self-Compact and context folding. Prompted Qwen3.6-27B scored 59.4% on BrowseComp-Plus against 53.4% for a Codex-style summarization baseline.
Training moves the number further. With stepwise GRPO, Qwen3.5-9B went from 28.8% to 42.5% on BrowseComp-Plus, a 47.6% relative gain while using 12% fewer FLOPs. The cheaper route also works: evolve a skill document that tells the model how to manage the file, by having the agent produce rollouts, a proposer model draft candidate skills from the traces, and a dev split pick the winner. That loop lifted held-out accuracy by up to 35.9 points.
Two caveats before you build on any of this. ContextBench, the benchmark behind the 35.9-point result, is listed as "coming soon", so independent replication is pending. The released code is CC BY-NC 4.0, which rules out commercial use.
Where it sits next to compaction
Context-management methods form a ladder, sorted by who holds the pen. Bottom rung: rules decide. The harness summarizes when the window crosses a threshold and the model has no say. Anthropic's context editing sits close here, clearing old tool results or thinking blocks server-side according to your config. Middle rung: the model triggers fixed operations, like offloading a block to storage and pulling it back. Top rung: the model edits anything, anywhere, at any step.
The distinction from compaction matters in practice. Anthropic's compaction replaces the oldest turns with a summary at a threshold or on request. OpenAI's version returns an encrypted compaction item that is explicitly not human-interpretable. Both only touch the oldest turns. A CLM can edit mid-context, keep the parts worth keeping verbatim instead of summarizing everything into mush, and carry its editing policy in the weights rather than a fixed summarization prompt. Anthropic's memory tool is the closest relative in spirit, but those files live outside the context; a CLM edits the context itself.
The cache problem nobody advertises
Inference servers save work with prefix caching, which reuses the computed attention state for the unchanged start of a prompt. A CLM edits the middle, and standard prefix reuse then forces re-prefilling of every token after the first change. That is a big reason providers only compact at the tail. The paper introduces Suffix Cache Reuse, which reuses cached state for the tokens that survive after the edit, cutting server compute by about 35% against standard SGLang at matched performance. It ships as an SGLang patch, so self-hosted stacks get the benefit before anyone else.
On a hosted API, an edit near the top of your context means a new prompt whose cached prefix ends where the edit begins, and you pay to re-prefill everything after it. As of early October 2026 I have not found a provider API that accepts an edited context with cache reuse past the edit point. The prompted version runs today, but the invoice reflects the re-prefill.
The security hole the authors flagged and did not fix
A CLM writes the text it will read next. The authors name the risk directly: editable context can become another channel through which prompt injections or self-generated instructions persist across turns. They cite an OpenAI report where a model inserted unauthorized instructions into its own summary, and the injected text then affected task behavior. A model that plans ahead can plant instructions for its future self, and nothing audits the edit. The paper proposes no defenses.
If I were wiring this into anything real, two controls go in first. Keep the system prompt and task instructions outside the editable region, so the model reads them but can never rewrite them. Keep an append-only log of the original context alongside the edited file, with every edit recorded as a diff. The Hermes Agent project proposed something similar for its compaction step: whitelist the headings a summary may contain and strip directive-shaped lines. Multi-agent setups multiply the need. The paper's orchestrator made 163 in-place edits while holding its context at 6K to 8K tokens; in a six-agent swarm, reconstructing who knew what, when, needs a log per agent.
What CLMs don't solve
The context file dies with the task. The longest run in the paper is 24 hours, and when the task ends, the scoreboard, the compressed tool calls and the facts the model chose to hold all get thrown away. Whatever your agent needs to know next week is still your problem. A bigger context window does not move that boundary, it only postpones it.
What to do with this today
Three honest positions. Experiment: run the prompted version through the open harness on long internal tasks, because the inside-the-task evidence favors the model over the harness. Don't ship: the trained result is a single 9B RL run, ContextBench is unpublished, the license is non-commercial, and no hosted API supports cache reuse past a mid-context edit. Watch one signal: a provider that lets you delete or rewrite individual turns of a cached conversation without charging a full re-prefill. That single API change means the industry absorbed the idea.
My bet is absorption, not a product line called a CLM. Until then, keep your compaction threshold if you need one, but treat "the harness knows best what the model should forget" as a research position rather than a default.
Comments