An agent that forgets what it was doing three tool calls ago doesn't have a model problem. It has a context problem.
Context engineering decides what information reaches a model's context window at each step of a run, and how that information is shaped once it arrives. Prompt engineering shapes how a single instruction is phrased, a narrower job that context engineering subsumes as one part of a larger whole.
This guide explains four context-engineering patterns and the infrastructure each one requires.
Copy link to headingKey takeaways
Context engineering decides what enters a model's context window and how it's shaped at each step of a run, not how a single instruction is worded.
More context can make results worse because retrieved or remembered content increases the likelihood that irrelevant or contradictory information reaches the model alongside what it actually needs.
There are four core patterns: retrieval, memory, compression, and tool context. Each keeps a different part of an agent's working state usable, and a given task might need one, several, or all four.
A router that collapses under contradictory sources, such as two versions of the same guideline, can produce an answer without flagging that a conflict existed at all.
Every one of the four patterns runs on infrastructure: storage for retrieved data, compute for long loops, a routing layer for token spend, and typed schemas for tool calls.
Copy link to headingWhat is context engineering?
Context engineering is the discipline of deciding what fills a model's context window for the next step of a task, and in what form. A 2025 survey formalized the split this way: prompt engineering optimizes a single instruction, while context engineering treats the context window as a structured, assembled payload that gets rebuilt on every step of a run, not a static string written once.
That distinction matters most in a multi-step agent loop, where the context window accumulates tool outputs, retrieved documents, and prior decisions as the run progresses. Each iteration reassembles that state and appends whatever the last step produced, so the mix inside the window keeps changing even when the task hasn't.
What actually occupies that window breaks down into a few categories:
System instructions: The stable part of the payload, present on every call and rarely changed mid-run.
Tool definitions: Structured metadata traveling with each request, including a tool's name, description, and parameter schema, which the model reads before deciding whether to call it.
Conversation history: The turn-by-turn record of the run so far, growing with every exchange unless something actively trims it.
Tool results: Output from whatever the agent has already executed, varying widely in size depending on what the tool returned.
Retrieved documents: Content pulled in per step to answer the immediate question, present only when that step actually needed outside information.
None of these categories sits still. Every one of them needs active curation on each loop iteration, which is the job context engineering exists to do.
Copy link to headingHow context engineering differs from prompt engineering
Prompt engineering shapes a single instruction: phrasing, examples, and structure that make one call perform well. Context engineering operates one level up. It manages the information supply across an entire multi-step run, which makes prompt engineering one component of the larger discipline rather than a competing one.
The distinction is clearest when a well-written prompt receives conflicting context. A 2026 framework for detecting knowledge conflicts in retrieval-augmented generation tested this directly: removing conflict detection from an otherwise identical pipeline dropped answer correctness by 16 percentage points, because the system fell back to concatenating contradictory documents and generating from whichever source happened to dominate, with no signal to the reader that a conflict existed at all.
No amount of prompt refinement can fix that failure, because the problem lies in what entered the window, not in how the question was asked. Better control over what reaches the model is what closes the gap.
Copy link to headingWhy capable agents depend on context engineering
A model's attention is a limited resource, and how it gets spent determines whether an agent stays capable as a task grows. A few mechanics explain why:
Attention is finite: every token added to the context window draws on the same fixed budget, so more context doesn't automatically produce a better answer; it can as easily mean more competition for that budget.
Attention is pairwise: a transformer weighs every token against every other token, so doubling the tokens in a window not only doubles the information available; it also multiplies the relationships the model has to sort through to find what matters.
Recall degrades gradually, not suddenly: accuracy drops as token count climbs well before a model hits its hard context limit, a pattern often called context rot, and it shows up across model families rather than one architecture.
Training skews toward shorter sequences: models see fewer long-range dependencies during training than short ones, so they have less practice weighing information that's far apart in a long context, even when that context technically fits.
None of this means agents can't handle long or complex tasks. It means the fix is keeping what enters that window high-signal, which is exactly what the four patterns below are built to do.
Copy link to headingCore context engineering patterns
These four patterns operate independently, and a given step in a task might need one, several, or all four depending on what it's missing: outside knowledge, durable state, a smaller working set, or a clearer tool interface.
Copy link to heading1. Retrieval
Retrieval pulls in information the model doesn't already have, at the moment a step actually needs it. Setups vary by corpus and task: a basic setup passes raw top-k vector matches straight into the window, while a ranked setup retrieves a wider candidate set, scores it for relevance, and passes only the highest-ranked chunks through.
The basic version is also where retrieval most often goes wrong. A fixed high top-k brings in chunks that match superficially, and irrelevant passages degrade an answer even when the correct chunk is sitting right there beside them. Enforcing a minimum relevance score before a chunk enters the window keeps generation focused, particularly when the corpus contains overlapping or outdated sources.
Copy link to heading2. Memory
Where retrieval supplies what the model never had, memory carries forward what the run has already produced, within a session and across sessions. Externalized notes write task state outside the active window and pull it back in later, which matters because short-term memory is bounded by the context window itself and doesn't survive past the current session on its own.
A long-running agent that stores decisions as notes can avoid re-deciding something it already settled two steps ago. That only holds if the notes are curated on the way in, because anything written to memory gets treated as ground truth the moment it's read back, contradictions included.
Copy link to heading3. Compression
Memory moves the state outside the window. Compression keeps what remains inside usable as it grows. The NoLiMa benchmark measured this directly: at 32,000 tokens of context, 11 of the 13 evaluated models dropped below half their short-context baseline accuracy, and GPT-4o, one of the stronger performers in the study, fell from 99.3% to 69.7%.
Compaction is the practical response: summarize a conversation nearing its limit, then reinitiate the run with the summary in place of the full history. A summarization pass that discards redundant tool output while preserving open questions and unresolved decisions keeps the signal that matters. The failure mode runs the other way: a compacted summary that drops a constraint or an open question, along with the redundant detail, hands the next step a smaller, faster, and quietly wrong context.
Copy link to heading4. Tool context
The first three patterns manage what the model knows. Tool context manages the interface it acts through, and that interface is context too. Every tool definition includes a name, a description, and a parameter schema that the model must interpret correctly before it acts.
A vague or incomplete tool description can make the wrong tool look like the right one, and ambiguous parameter names compound the problem when a model has to guess which field means what. Unambiguous naming (user_id instead of user), strict schema validation, and loading only the tools a given step actually needs all reduce how often a model calls the wrong thing or supplies an invalid argument.
Copy link to headingHow the context engineering patterns compare
The table below summarizes what each pattern keeps stable, which is the fastest way to decide which one a given step is missing:
All four patterns share one underlying dependency: infrastructure. Retrieval needs somewhere to store and serve data close to where the agent runs; memory needs compute that outlives a single request; compression needs a place in the loop to compact state before the next call; and tool context needs a runtime that enforces the schema it defines.
Copy link to headingWhere Vercel fits in a context engineering setup
Whether these four patterns hold up under real traffic depends on what's running underneath them.
Copy link to headingServing retrieval without the latency tax
An agent that retrieves mid-loop adds a data round trip to every step that needs it, and that latency compounds across a multi-step run. The general fix is to keep storage close to compute and to reach for a managed data store instead of standing up separate retrieval infrastructure, since smaller, better-ranked chunks reduce the tokens sent to the model at every step and a nearby store prevents non-model network hops from stacking up across a long loop.
Vercel Marketplace provides native integrations with storage providers, including managed Postgres via Neon and managed Redis via Upstash, so a team can connect to a data store from Vercel Functions within the same agent loop instead of routing retrieval through a separate service boundary.
Copy link to headingMemory that survives long-running tasks
A function that times out mid-run takes the agent's in-flight state with it, which is why any serverless execution model needs an explicit answer for state that must outlive a single request. The general principle is separating a duration ceiling on compute from a durable store for progress, so curated state, not the full history, is what carries forward between steps.
Vercel Functions on fluid compute run for 300 seconds by default across every plan, with active CPU pricing charging only for the time the function is actively executing, while time spent waiting on I/O is billed at the lower provisioned-memory rate instead. For state that needs to outlive that ceiling, Vercel Workflows is built specifically for tasks that pause, resume, and hold state for minutes to months, giving a long-running agent a durable place to carry progress between model calls, tool results, or a human review step.
Copy link to headingCompression, caching, and model routing
Every step's token bill climbs as retrieved and historical content accumulate, which is exactly what compression is meant to counteract before the next request goes out. Compaction works best when routing and context decisions sit close together in the same code path, since keeping the logic that picks a model next to the logic that decides which messages, summaries, or retrieved chunks enter the next request avoids a split where one side compresses aggressively while the other keeps routing to a model priced for the uncompressed version.
AI Gateway provides a team with a single endpoint across providers, with automatic failover and usage tracking that remain consistent regardless of which model handles a given call.
Copy link to headingTool context with typed definitions
A tool definition works best as a runtime contract rather than a prompt-writing concern, which means it belongs beside the code that executes the tool, not off in a separate file the runtime has to trust blindly. That keeps the model and the runtime reading the same schema, rather than two versions that can drift apart.
AI SDK gives TypeScript teams a shared way to define tools and their parameter schemas. The current tool() helper takes an inputSchema built with Zod, which the model consumes to understand the interface and the runtime uses to validate every call against it:
import { tool } from 'ai';import { z } from 'zod';
export const getOrderStatus = tool({ description: 'Look up the current status of a customer order by ID', inputSchema: z.object({ order_id: z.string().describe('The unique order identifier'), }), execute: async ({ order_id }) => {// fetch and return order status },});
The AI Gateway and AI SDK guide covers pairing typed tools with fallback models when a step needs both a validated interface and a backup provider.
Copy link to headingStart engineering your agent's context
The agent that forgets what it was doing three tool calls ago has a context-supply problem, and a longer window doesn't fix that. It gives uncurated tokens more room to sit in while the same four decisions: retrieval, memory, compression, and tool context, still have to be made deliberately at every step.
Vercel is the platform where those four decisions turn into running infrastructure: it hosts the application, runs the compute behind an agent loop, and provides the storage, functions, and model routing that keep each pattern in this guide working in production.
Each pattern maps to a concrete piece of infrastructure to get right:
Retrieval: Vercel Marketplace connects managed Postgres and Redis providers directly to Vercel Functions, keeping data close to the agent loop that calls it.
Memory: Vercel Workflows hold state for minutes to months, past what a single function's duration ceiling allows.
Compression: AI Gateway keeps model routing and cost tracking in one place as compaction decisions change what a request actually costs.
Tool context: AI SDK's typed
tool()definitions give the model and the runtime one schema to agree on, instead of two that can drift.Everything above: runs from a standard Next.js project without extra infrastructure to stand up first.
Start a new project at vercel.com/new or begin from a working example at vercel.com/templates to wire up a first typed tool today.
Copy link to headingFrequently asked questions about context engineering
Copy link to headingDoes a larger context window remove the need for context engineering?
No. Recall degrades as tokens accumulate well before a model hits its hard limit, a pattern sometimes called context rot. Windows of any size still need curation, since irrelevant or stale content crowds out what the next step actually needs, regardless of how much room is available.
Copy link to headingIs context engineering only relevant for AI agents?
No. The discipline applies to any production LLM application, including RAG pipelines and chatbots, since each one assembles context on every call. Agents raise the stakes because their windows grow across steps and accumulate more tool output and remembered or retrieved state than a single-call system ever sees.
Copy link to headingHow do you know if an agent has a context problem versus a model problem?
Trim the context and rerun the task. If quality recovers with a short, focused window, the model was working from the wrong working set, not failing on capability. Failures that correlate with input length or appear mid-task or immediately after a schema change point to context rather than the model itself.
Copy link to headingDoes context engineering require a specific framework?
No. The four patterns are decisions about what enters the window, and direct API calls can implement every one of them. LangChain and LlamaIndex supply primitives such as retrievers and memory stores, and AI SDK supplies typed tools, but the discipline itself transfers across whichever framework, or no framework, a team chooses.