Cache Control

Pruning solves one half of the problem. The other half is that every call still sends the parts of the context that didn't change.

Your system prompt is the same on call ten as it was on call one. The early user prompt is the same. The tool definitions are the same. The provider has seen these tokens before, and you're paying to send them again.

Cache control is the provider's way to say "I remember this, don't process it again." Where it's supported, the cost drop is real. Where it isn't, the pattern still teaches a useful habit: separate stable context from fresh context.

Outcome

A prepareStep pipeline that prunes old tool results, then marks cache breakpoints on what's left. On providers that support cache control, repeated input costs drop substantially.

Fast Track

1. Add an addCacheControl(messages) helper that sets cache breakpoints
2. Compose it after pruneMessages in prepareStep
3. Mark the first message and the last message, and leave the middle alone

Hands-on Exercise 5.4

Wire cache control behind the pruning step.

Requirements:

1. Write addCacheControl(messages) that returns a new array of messages with providerOptions.anthropic.cacheControl set where appropriate
2. The first message (the initial user prompt) always gets a breakpoint
3. The last message always gets a breakpoint
4. Every message in between stays unmarked
5. Update prepareStep to call addCacheControl(pruneMessages(...))

Implementation hints:

• cacheControl: { type: "ephemeral" } is the Anthropic-flavored shape. Other providers use different keys. If you're hitting one that doesn't support cache control, the call still works, the header just gets ignored
• The cache breakpoints work by prefix. Marking message 5 as cacheable caches everything up to and including message 5. The provider checks the prefix on the next call
• In an agent loop, new messages get appended to the end. Nothing gets swapped out, so the last message is the right place for a breakpoint: the next step's prefix starts with everything you just sent
• Anthropic caps how many breakpoints one request can carry (prompt caching docs). Mark a couple of messages, not every message

The helper

import type { ModelMessage } from "ai";
 
export function addCacheControl(messages: ModelMessage[]): ModelMessage[] {
  return messages.map((msg, i) => {
    if (i === 0 || i === messages.length - 1) {
      return {
        ...msg,
        providerOptions: {
          ...msg.providerOptions,
          anthropic: { cacheControl: { type: "ephemeral" } },
        },
      };
    }
    return msg;
  });
}

Two messages get the marker. The last one is where Anthropic recommends a breakpoint for a growing conversation. Each step appends to the end, so the next request starts with the exact prefix this one just wrote, and the provider reads it back from cache.

The first one is our insurance policy against pruning. pruneMessages drops old tool results from the middle of the history, and when the middle changes, the prefix behind the last breakpoint changes with it. The breakpoint on the first message caches the part pruning never touches: the tool definitions, the instructions, and the original prompt, which all come before it.

You might be tempted to mark every older message instead. Don't. Anthropic limits the number of breakpoints per request, and a long session blows past that limit fast.

Compose with pruning

In index.ts, the prepareStep from 5.2 becomes a pipeline:

import { addCacheControl } from "./src/cache";
 
prepareStep: ({ messages }) => ({
  messages: addCacheControl(
    pruneMessages({ messages, toolCalls: "before-last-3-messages" }),
  ),
}),

Pruning happens first because it changes how many messages there are. Caching happens second on whatever messages survive.

What the savings look like

The numbers depend on the provider, the cache hit rate, and how much of your context is stable. For a typical Anthropic-backed agent running long sessions:

That's an order-of-magnitude swing on long sessions. On short sessions, the difference is smaller because the cache doesn't have time to amortize. The discipline still applies.

Why the pattern matters even where caching isn't supported

You're forcing yourself to think about which parts of the prompt are stable. That's a useful habit. It tells you whether your system prompt is doing too much work per-call (rebuilt every time) or just right (built once, reused).

If caching disappeared tomorrow, the discipline of identifying stable context would still be worth keeping. Stable context is also easier to test, easier to version, and easier to reason about than context that changes shape every call.

Try It

Run a multi-step task and watch the token counts:

bun run index.ts . "Read package.json, tsconfig, index.ts, then summarize"

If your provider returns cache hit info in the usage object, log it:

onStepEnd: ({ usage, stepNumber }) => {
  console.error(
    `Step ${stepNumber}: ${usage.inputTokens} input, ${usage.outputTokens} output, ${usage.inputTokenDetails.cacheReadTokens ?? 0} cached`,
  );
},

Step 0 writes the cache, so its cached count is 0. From step 1 onward you should see inputTokenDetails.cacheReadTokens above zero, covering the stable prefix the earlier steps wrote.

npx tsc --noEmit

Commit

git add src/cache.ts index.ts
git commit -m "feat(context): add cache control to stable messages"

Done-When

• addCacheControl(messages) marks the first and last messages, and nothing else
• prepareStep runs pruning then caching, in that order
• On a provider that supports caching, inputTokenDetails.cacheReadTokens shows up in usage
• npx tsc --noEmit passes

Solution

import type { ModelMessage } from "ai";
 
export function addCacheControl(messages: ModelMessage[]): ModelMessage[] {
  return messages.map((msg, i) => {
    if (i === 0 || i === messages.length - 1) {
      return {
        ...msg,
        providerOptions: {
          ...msg.providerOptions,
          anthropic: { cacheControl: { type: "ephemeral" } },
        },
      };
    }
    return msg;
  });
}

prepareStep: ({ messages }) => ({
  messages: addCacheControl(
    pruneMessages({ messages, toolCalls: "before-last-3-messages" }),
  ),
}),