A developer’s framework for building AI agents that survives production chaos

A prototype agent that answers questions in a demo and a production agent that survives real traffic are built from the same model calls, but almost none of the same decisions. The gap between them is scope, tooling, instructions, and the guardrails that keep a wrong action cheap to undo.

Most of that work happens before a single request hits production. This guide covers the decisions that determine whether an agent is ready to carry real traffic, and where Vercel's runtime picks up once it does.

Key takeaways:

• AI agents are well-suited to tasks where the path changes by request, the system needs external context, and mistakes are cheap to reverse.

• Production readiness starts with measurable success criteria, scoped tools, clear instructions, and evals that prove the agent handles real traces, not synthetic ones.

• Tool permissions, approval gates, loop ceilings, and cost tracking keep agent autonomy bounded without removing its flexibility.

• Vercel supports agent workloads with fluid compute and Vercel Workflows for runtime, the AI SDK for tool definitions and approval flows, AI Gateway for model access, and Vercel Sandbox for isolated code execution.

Copy link to headingWhat is an AI agent?

An AI agent is a system built on a large language model that selects its own steps and tools to pursue a goal over repeated iterations. It decides what to try, calls a tool, reads the result, updates its plan, and continues until it finishes or needs a person to decide.

Control is what separates an agent from a workflow. A workflow keeps execution in your code, calling the model at fixed points along a path you defined in advance. An agent moves that decision-making into the model and the loop around it, so the same request can take a different number of steps depending on what it finds along the way.

Copy link to headingWhat’s the difference between AI agents, workflows and chatbots?

The three control models produce different behavior across every dimension that matters in production, and the differences show up before day one, not after:

A chatbot follows a script and hallucinates the moment a question falls outside it. A workflow follows your code and fails silently when the input doesn't match what you planned for. An agent follows its own assessment of the goal, which gives it greater reach across varied inputs and a wider surface for a small mistake to compound.

Copy link to headingWhen should you build a production-ready AI agent?

Build an agent when the path cannot be known in advance, and use a workflow whenever possible. Every extra loop and tool call adds latency, cost, and another place for a small mistake to compound, so agentic control has to earn its place rather than arrive as the default.

Five conditions settle it, and the last is the one most agent guidance skips:

1. The step count changes by request: A fixed sequence would fail on enough requests that maintaining the branches by hand stops being practical.

2. The next action depends on what retrieval returns: the system cannot pick a path until it has fetched and read something, which is the decision a model in a loop exists to make.

3. A wrong action is cheap to undo: Agent flexibility is only safe where recovery is cheap, so anything expensive to reverse belongs behind an approval gate rather than inside the loop.

4. The task can absorb the added latency: Each iteration costs a round trip, and an interactive surface feels that accumulation long before a batch job does.

5. A deterministic pipeline wins whenever the path is already known: When every relevant fact is available up front, and the sequence repeats identically, a workflow returns the same result with lower latency, lower spend, and no drift.

The deciding function underneath all five is error probability multiplied by recovery cost, which is why that inverse case rules out more candidate agents than the other four conditions combined. A practical middle ground embeds an agent as a bounded component inside an otherwise deterministic workflow, keeping autonomy scoped to the one step that needs it.

Once a task clears that bar, the build order decides the rest.

Copy link to headingHow to build an AI agent step by step

Define success criteria first, because they settle the arguments in every step that follows.

Copy link to headingDefine the job and success criteria

Decide which task the agent owns and what "done" means in measurable terms. A narrow scope ships sooner but covers less ground. A broad scope invites compounding errors across steps a team can't yet evaluate.

Filter the scope by recovery cost. The agent should own work where a wrong action is cheap to undo, while operations that are expensive to reverse stay behind an approval gate. The success criteria written now become the eval set later, which is why skipping this step gets expensive down the line.

Copy link to headingChoose your model

Choose a model by weighing capability against cost, in that order. First, prove the task can be solved with a strong model on the eval set; then test smaller or cheaper models where the quality holds. Leaderboards and provider pricing guide the first pass, but neither proves a model will hold up inside a specific agent loop and traffic mix.

The surrounding loop changes how a model behaves in practice, so a leaderboard result alone isn't a substitute for testing inside the actual system. Defaulting every step to the largest available model carries its own cost, because high-capability models dominate spend when they handle every step rather than only the ones that need them.

AI Gateway makes the alternative cheap to test, reaching models from multiple providers through one key, so routing a step to a different model is a string change, with failover when a provider degrades and latency, token, and spend reporting per provider in one place.

Copy link to headingDefine the agent's tools

Separate context retrieval from side-effecting actions, and add orchestration tools only when one agent has to call another. Expose only the surface the agent needs, and describe each tool as if it were being handed to a teammate seeing it for the first time, with unambiguous parameter names, such as user_id instead of user. A vague tool boundary makes a wrong call more likely.

With the AI SDK, a tool is a typed interface, and the schema gives the model a bounded contract for the call:

import { z } from 'zod';
import { tool } from 'ai';

export const weatherTool = tool({
  description: 'Get the weather in a location',
  inputSchema: z.object({
    location: z.string().describe('The location to get the weather for'),
  }),
  execute: async ({ location }) => ({
    location,
    temperature: 72 + Math.floor(Math.random() * 21) - 10,
  }),
});

Overlapping or vaguely described tools raise the odds of a wrong call or malformed arguments, and they can duplicate side effects when the model retries. Action tools need to be idempotent so a retry has a safe target to land on.

Copy link to headingWrite the instructions

Write instructions for inputs that a team didn't anticipate, since those are most of the production traffic. Terse instructions leave the model improvising on edge cases, while exhaustive ones bloat context and still miss situations they didn't plan for. Spell out explicit actions for known cases, and spell out when to stop and ask instead of guessing, including how to handle incomplete input.

Teams often discover, only after behavior drifts, that the model was never told the preference they expected it to follow. Tool descriptions and system instructions are part of the product surface as much as any configuration file, so fix the words first when behavior goes wrong.

Copy link to headingChoose one agent or several

Start with one capable agent. Additional agents make coordination and debugging harder and increase token usage, so splitting should follow evidence rather than architectural preference.

Split only when evals show one of two failures: either the agent can't reliably follow complex instructions or it confuses similar tools across tasks. If the evidence supports it, a handoff or orchestrator-worker pattern is easier to evaluate than custom coordination logic, though the decision of whether to split at all comes first.

Copy link to headingWrap it in guardrails and approval gates

Before any of this touches production, wrap the agent in guardrails and approval gates. OWASP's Excessive Agency entry names three root causes: excessive functionality, excessive permissions, and excessive autonomy, and points to hallucination and prompt injection as the triggers that turn them into damage.

In the AI SDK, setting needsApproval: true on a tool pauses execution until a person confirms the action; the full pattern is covered in the human-in-the-loop cookbook. Place validation next to the tool that creates the side effect, so it runs everywhere that tool runs.

These gates are what keep the agent working once production traffic replaces the synthetic test cases it was built against.

Copy link to headingRequirements for building production-ready AI agents

Four supports carry production readiness. An eval set, scoped permissions, loop ceilings, and cost tracking each turn a build decision into something a team can verify instead of assume.

Copy link to headingBuild an eval set before you scale

A prompt tweak improves one case, and without an eval set there's no way to know what it regressed elsewhere. An eval set turns that guesswork into evidence. Start with a small set of examples drawn from real production traces, then keep adding cases as new traces surface new failure modes.

The set should grow toward a CI regression suite that covers core features and past bugs, including edge cases. Deterministic assertions are preferable wherever the task allows them, and the suite should run before every deploy.

Copy link to headingScope tool permissions and gate irreversible actions

Give each tool only the access it needs to do its job. Granular tools beat open-ended shell runners, and any action that sends, deletes, publishes, or moves money should require approval before it executes.

This keeps autonomy tied to intent. Minimum permissions paired with approval-gated, idempotent action tools give the agent room to work while keeping the important side effects under a team's control.

Copy link to headingSet loop and cost ceilings so a stuck agent stops itself

Retry loops need explicit stopping conditions rather than relying on the model to detect when it's stuck. A predictable agent loop has a step cap, a tool-call cap, a token cap, a wall-clock timeout, and a cost ceiling, all tracked outside the loop through loop control rather than left to instructions the agent can choose to ignore.

Those ceilings should match the task's recovery cost. A research assistant can tolerate a higher step count than an agent with access to publishing or payment tools. The eval set should include cases that prove the run stops cleanly when it hits a ceiling, not only cases that prove it succeeds.

Copy link to headingTrack cost by completed run

A single business outcome can span planning calls, retrieval, tool retries, and memory writes, so per-request cost tracking underestimates the actual cost of a task. Track spend per agent_run_id, and separate the typical operating range from the high-percentile outliers.

The typical run tells a team whether the unit economics work. The tail shows which loops need tighter caps or model routing changes. That view also gives engineering and finance the same unit of measurement, the completed task rather than the individual API call.

Copy link to headingHow Vercel helps AI agents survive production

Everything above determines how much autonomy an agent should carry. Production adds a separate set of runtime questions about where the loop executes, how state persists across turns, how progress reaches the UI, and where generated code runs. Getting those right is what makes an agent behave like an operational service rather than a demo.

Copy link to headingRunning loops that outlive a request on Fluid compute

An agent that requires eight sequential tool calls must have sufficient runtime to complete the work and sufficient structure to record progress along the way. Fluid compute runs functions for 300 seconds by default, configurable up to 800 seconds on Pro and Enterprise plans, with an extended 1,800-second duration available in beta for supported runtimes.

Billing matters as much as duration. Active CPU pricing pauses charges during I/O wait, and an agent loop spends much of its wall-clock time waiting on model responses. Post-response work, such as logging, can run after the response is sent via after() in next/server on Next.js 15.1 and later, or via waitUntil in @vercel/functions elsewhere.

Copy link to headingPersisting run state with Vercel Workflows

A long-running agent that records progress after each step can resume without replaying work it already did, and a stateless function gives it nowhere to record that progress between invocations.

Vercel Workflows supplies the missing half. Marking a function with 'use workflow' makes each step durable, so a run can pause for minutes or months, resume from the exact point, and survive a deploy through deterministic replay, with managed persistence that holds state and event logs rather than a database the team operates.

Copy link to headingStreaming reasoning to the UI with the AI SDK

A multi-step run is easier to trust when the interface shows progress while the agent works, rather than waiting silently for a final response. The AI SDK's streamText streams model output and tool activity so the UI reflects each stage of the loop as it happens.

On the client, useChat exposes messages in a structure that the UI can render as the run progresses, and an approval flow can appear in the same interaction when a gated action requires a human decision.

Copy link to headingIsolating agent-generated code with Vercel Sandbox

Code that an agent wrote is untrusted by definition, so it should avoid using a deployment's environment variables and API keys. Vercel Sandbox runs untrusted code in Firecracker microVMs, giving each run its own kernel, filesystem, and network without touching production credentials.

That isolation gives a team a clean boundary between generated code and production secrets. It also keeps code execution as a bounded tool call, separate from the deployment environment where the rest of the application runs.

Copy link to headingShip your AI agents from prototype to production

The distance between a prototype that works and an agent that holds up under real traffic comes down to how deliberately a team settled fit, model choice, tool scope, splitting strategy, evals, permissions, and cost ceilings. Each one determines how much autonomy the system can safely carry and how expensive it is to walk back a mistake.

Vercel closes the runtime side of that work:

• fluid compute and Vercel Workflows: Long-running loops with active-CPU pricing during I/O wait, plus durable orchestration for runs that need to pause and resume past a single request.

• AI SDK: Typed tool definitions, needsApproval for human-in-the-loop gating, and the ToolLoopAgent abstraction for reusable agents across a chat UI, background job, or API route.

• AI Gateway: Model access with usage visibility across providers, so routing and cost decisions don't require a separate vendor integration.

• Vercel Sandbox: Isolated microVMs for running agent-generated code away from production environment variables and credentials.

Start a new project at vercel.com/new to put these primitives into practice, or browse vercel.com/templates for agent examples already wired up for production.

Copy link to headingFrequently asked questions about how to build AI agents

Copy link to headingWhat does it cost to run an AI agent in production?

Cost varies by loop depth, model choice, tool retries, retrieval volume, and the frequency with which a run escalates to a more expensive model. Because a single completed task can span planning calls, retrieval, retries, and memory writes, tracking spend per completed run provides a more accurate picture than tracking it per API call.

Copy link to headingHow long does a first production AI agent take to build?

Scope the first version as a single agent before adding multi-agent coordination if speed matters. Enterprise timelines depend more on integrations, governance configuration, security review, and the eval coverage required before launch than on the agent logic itself.

Copy link to headingDo you need a framework to build an AI agent?

No. Core agent patterns can be implemented directly with LLM APIs and a small amount of orchestration code. A framework adds an abstraction layer that can hide details a team needs when debugging, so adopt one only once the orchestration or checkpointing it handles has earned that tradeoff.

Copy link to headingHow do you know an AI agent is ready to ship?

An agent is ready when evals and logs prove its behavior, not when it looks correct in a demo. Evals should pass on task and safety suites with real path coverage, tool permissions should be scoped, approvals should gate irreversible actions, and cost should be bounded per run. Before launch, review real traces, failed eval cases, approval logs, and high-cost outliers together rather than in isolation.