Claude Message Threads: 35% Faster First Token, Same Token Bill

Claude Message Threads: 35% Faster First Token, Same Token Bill

Every agent loop built on a chat API shares one dumb property. The model is stateless, so each turn your client re-uploads the whole conversation: the system prompt, the tool definitions, every earlier tool result, every turn since lunch. By the time a coding session accumulates a few hundred thousand tokens, you push a megabyte of JSON over the wire just to ask the next question. Anthropic shipped a fix for the wire part of that problem on November 2, 2026. It is called message threads, it is in public beta on the Messages API, and the headline numbers are easy to misread.

The company says median time to first token drops about 35% on requests between 500K and 1M tokens. Anthropic engineer Blake Gentry frames the payload change in starker terms: median request size in Claude Code sessions went from roughly 800 KB to 1.8 KB, a cut of about 99% in upload bytes. Both numbers are real. Neither one means what people are already taking it to mean. Here is what actually changes, and where the edges are.

What threads change

The first request in a thread works the old way: full conversation, system prompt, tool definitions. That request creates server-side state. Every request after it can carry only the new messages plus a pointer, the id of Claude's previous response. The API keeps the earlier conversation on its side, so your upload shrinks to almost nothing. The model still reads the full conversation every turn. You just stop mailing it a copy.

That is the whole feature. It is a transport fix, not a memory feature. The latency win comes from two places: fewer bytes crossing the network, and less server-side parsing of a giant request body before the first token comes back. The 35% is a median on long requests, so short sessions barely notice.

The mechanics

Because it is beta, every request in a thread needs the header anthropic-beta: message-threads-2026-08-12. The raw API takes a new thread parameter. The first call creates the thread:

{
  "model": "claude-opus-5-5",
  "max_tokens": 1024,
  "system": "You are a support agent for Acme.",
  "messages": [ ...full history... ],
  "thread": { "type": "create" }
}

Follow-up requests are short:

{
  "model": "claude-opus-5-5",
  "max_tokens": 1024,
  "messages": [
    { "role": "user", "content": "How long does standard shipping take?" }
  ],
  "thread": { "type": "continue", "previous_message_id": "msg_01..." }
}

One subtlety worth internalizing: the creating request locks in the system prompt and tool definitions. A continue request that tries to change them has nowhere to put the change. If your agent adds tools mid-session, and plenty of frameworks do exactly that, you want a new thread, not a new turn.

The SDKs hide the bookkeeping. The Python helper creates the thread on the first send and continues it after that, while keeping a local copy of the history:

client = anthropic.Anthropic()

thread = client.beta.messages.thread(
    model="claude-opus-5-5",
    max_tokens=1024,
    system="You are a support agent for Acme.",
)
thread.send({"role": "user", "content": "Hi, I have a question about an order."})
reply = thread.send({"role": "user",
                     "content": "How long does standard shipping take?"})

What threads do not change

The bill. Claude still reads the full conversation on every turn, so input tokens, output tokens, and rate limits are identical to the non-threaded path. Anthropic's own launch post says the gain is latency, not spend. Anyone telling you this makes agents 35% cheaper stopped reading at the first number.

Threads also do not change what the model knows. This is not long-term memory. When the thread dies, the conversation lives in your client or nowhere at all, which brings us to the part that deserves the most design attention.

The five-minute clock

A thread expires five minutes after its last request and is deleted within an hour of last use even if it stays active. It lives in encrypted memory, never written to disk, with no retrieval endpoint. There is no API call that reads a thread back. Use an expired or invalid thread id and the API returns 404.

So the rule is: your client remains the source of truth. Treat the thread as a cache, not as storage. The pattern that survives contact with reality looks like this:

1. Keep the full local transcript, exactly as you do today. Do not delete it because the server "has it."
2. Send continue requests while the thread is warm.
3. On a 404, create a fresh thread with a full replay and continue from there.

try:
    reply = send({"type": "continue",
                  "previous_message_id": last_id},
                 new_messages)
except NotFoundError:
    reply = send({"type": "create"},
                 full_transcript + new_messages)
    last_id = reply.id

The SDK helper does that fallback for you, which is the right default and also a trap if you assumed the server copy was authoritative. The failure mode to plan around is the slow agent: a turn with a long tool call or a human approval gate can blow past five minutes, and then the next request pays full replay anyway. Threads help loops that move fast. The agent waiting twenty minutes for a code-review approval gets nothing.

Compliance is a hard edge

Threads are not eligible for zero-data-retention agreements or HIPAA workflows. That is not a paperwork lag, it is the design: the conversation sits in someone's memory for a while, encrypted. If your contract forbids the provider holding content even briefly, threads are off the table. Check that before wiring them into a shared client library. Adopting the feature is a one-line change; ripping it out of a compliance-reviewed service is not.

Where it pays

Long agent loops, obviously: coding agents, computer-use agents, anything where tool results pile up over dozens of turns. Anthropic already defaults it on for most Claude Code users. The less obvious beneficiary is the network path between agents. Anthropic's Katelyn Lesse argues that bandwidth between agents starts to matter as multi-agent systems get more complex, and the math agrees: five subagents sharing a big context, each re-uploading history every turn, multiply an annoying cost by five. Distance helps too. The 800 KB to 1.8 KB cut matters a lot more on a mobile uplink or a far-flung edge function than on a datacenter link.

Availability is broad for a beta: every current Claude model, the Claude Platform, Claude Platform on AWS, Claude on Google Cloud (Anthropic managed), and Claude in Microsoft Foundry.

When to skip

Short sessions, under ten turns or well under 100K tokens: the latency delta is noise. Single-shot calls have nothing to continue. ZDR and HIPAA deployments cannot use it at all. And if your agent loop is really a state machine where each turn swaps tools, system prompts, or models, the locked-in creating request will fight you every time.

My read: this is plumbing, and good plumbing. It does not make agents smarter or cheaper. It makes the boring part of a long session less annoying, which is exactly the sort of feature that quietly becomes load-bearing. Adopt it where the loop is long and fast, keep your own transcript, and stop quoting the 35% figure to anyone who might hear "35% cheaper."