--- title: Build AI agents with AI Gateway and AI SDK description: Build AI agents on Vercel with AI Gateway and AI SDK, then make them reliable, capable, and durable with Sandbox, Chat SDK, Vercel Connect, and Workflow. url: "https://vercel.com/kb/guide/ai-gateway-and-ai-sdk" published: 2026-06-17 last_updated: 2026-09-14 authors: Ben Sabic related_resources: - title: "Software factory" url: "https://vercel.com/templates/eve/eve-software-factory" description: "Software factory built on eve: AI agents work each stage of the development loop, and people make the judgment calls." - title: "Incident response" url: "https://vercel.com/templates/eve/eve-incident-response-agent" description: "sre investigates production issues using a hypothesis-driven approach and outputs verifiable evidence from sources." - title: "Personal agent" url: "https://vercel.com/templates/nuxt/eve-personal-agent" description: "A durable AI assistant with long-term memory. Chat on the web or Slack, query Linear, and pick up where you left off." install_vercel_plugin: npx plugins add vercel/vercel-plugin --- Building an AI agent usually means managing a separate account, API key, and rate limit for every model provider you want to try. With the AI SDK and AI Gateway, a single endpoint and set of credentials can access hundreds of models, and switching providers is a one-string change. Once the agent works, Vercel Sandbox runs the code it writes in isolated microVMs, Chat SDK brings it to dozens of platforms, Vercel Connect gives it scoped access to third-party APIs through short-lived tokens, and the Workflow SDK keeps it running through crashes and timeouts. ## Overview In this guide, you'll learn how to: - Set up a [Next.js](https://nextjs.org/) project and authenticate to AI Gateway with OIDC tokens - Generate text, stream responses, and produce structured outputs - Give your agent tools so they can act, not just respond - Keep your agent available with model fallbacks - Run AI-generated code safely in isolated [Vercel Sandbox](https://vercel.com/sandbox) microVMs - Bring your agent to Slack, Teams, and other chat platforms with [Chat SDK](https://chat-sdk.dev/) - Give your agent scoped access to third-party APIs with [Vercel Connect](https://vercel.com/connect) - Make your agent durable and resumable with the [Workflow SDK](https://workflow-sdk.dev/) ## Prerequisites Before you begin, make sure you have: - A [Vercel account](https://vercel.com/signup). - [Vercel CLI](https://vercel.com/docs/cli) installed (`npm i -g vercel`) - Node.js 22+ and a package manager (e.g., [pnpm](https://pnpm.io/)) ## How it works AI Gateway is a single endpoint that sits in front of every supported provider. You send it a model string in the form `creator/model-name`, and the Gateway resolves the provider, authenticates, routes the request, and tracks usage. AI Gateway is the default provider in AI SDK 7, so a plain model string routes through it with no configuration, and your application code stays the same whether you call GPT-6 Astra, Claude Fable 5.1, or Gemini 3.8 Flash. Tokens cost the same as they would from the provider directly, with no markup. The AI SDK provides the function-level API you'll build the agent from, and AI Gateway provides the infrastructure underneath: authentication, usage tracking, failover, and billing. The two have high cohesion but loose coupling, so you can adopt the SDK on its own and add Gateway features via provider options. You'll use three core AI SDK functions in this guide: | Function | Returns | Use it for | | ---------------- | --------------------------------------------- | ---------------------------------------------------------------- | | `generateText` | Single text response | One-shot generation, background jobs, and agent loops with tools | | `streamText` | Streamed response | Chat interfaces and responses too long to wait for | | `generateObject` | A typed object validated against a Zod schema | Structured data extraction and machine-readable outputs | ## Steps ### 1\. Create a Next.js app Use `create-next-app` to bootstrap a new project: **pnpm** ```bash pnpm create next-app@latest ai-gateway-demo --yes ``` **npm** ```bash npx create-next-app@latest ai-gateway-demo --yes ``` **yarn** ```bash yarn create next-app@latest ai-gateway-demo --yes ``` **bun** ```bash bun create next-app@latest ai-gateway-demo --yes ``` The `--yes` flag uses the recommended defaults: TypeScript, Tailwind CSS, ESLint, App Router, and Turbopack, with the `@/*` import alias. Omit the flag if you want to customize these options interactively. ### 2\. Install the AI SDK Add the `ai` package to your project: **pnpm** ```bash pnpm i ai ``` **npm** ```bash npm i ai ``` **yarn** ```bash yarn add ai ``` **bun** ```bash bun add ai ``` This installs AI SDK 7, which uses AI Gateway as its default provider. If you're on an earlier major version, review the [AI SDK migration guides](https://ai-sdk.dev/docs/migration-guides) before upgrading, and note that the `WorkflowAgent` used later in this guide requires v7. ### 3\. Authenticate with OIDC AI Gateway authenticates requests using a [Vercel OIDC token](https://vercel.com/docs/ai-gateway/authentication-and-byok/oidc), which is automatically generated for your project. First, link your local directory to a Vercel project: ```bash vercel link ``` Then pull the [development environment variables](https://vercel.com/docs/environment-variables#development-environment-variables): ```bash vercel env pull ``` This writes the variables to your local environment file. The OIDC token will be included, and it will be valid for 12 hours. You'll need to run `vercel env pull` again to refresh the token when it expires. ### 4\. Generate text Start with the simplest request: generate a single block of text. Create an API route at `app/api/chat/route.ts`. Pass a plain string model ID to `generateText` and the AI Gateway resolves the provider and routes the request. With OIDC authentication, you don't reference a key anywhere in your code: ```typescript import { generateText } from 'ai'; export async function GET() { const { text } = await generateText({ model: 'openai/gpt-6-astra', prompt: 'Explain quantum computing in one paragraph.', }); return Response.json({ text }); } ``` Start the dev server and visit the route to see the response: ```bash pnpm dev ``` ### 5\. Stream responses For real-time output, use `streamText` and return a streamed response. This is the pattern you'll use for chat interfaces and any response long enough that waiting for the full result would hurt the experience: ```typescript import { createUIMessageStreamResponse, streamText, toUIMessageStream } from 'ai'; export async function POST(request: Request) { const { prompt } = await request.json(); const result = streamText({ model: 'anthropic/claude-fable-5.1', prompt, }); return createUIMessageStreamResponse({ stream: toUIMessageStream({ stream: result.stream }), }); } ``` ### 6\. Generate structured outputs Use `generateObject` with a [Zod](https://zod.dev/) schema to get type-safe structured data: ```typescript import { generateObject } from 'ai'; import { z } from 'zod'; export async function GET() { const { object } = await generateObject({ model: 'google/gemini-3.8-flash', schema: z.object({ name: z.string(), age: z.number(), city: z.string(), }), prompt: 'Extract: John is 30 years old and lives in NYC.', }); return Response.json(object); // { name: 'John', age: 30, city: 'NYC' } } ``` ### 7\. Give your agent tools So far, the model has produced text and data, but it hasn't done anything. Tools change that. You define functions that the model can invoke to fetch data, call an API, or act on the outside world. The AI SDK then runs the model in a loop, calling your tools and feeding each result back until the task is done. Define a tool with a description, an input schema, and an `execute` function: ```typescript import { generateText, tool } from 'ai'; import { z } from 'zod'; export async function GET() { const { text } = await generateText({ model: 'openai/gpt-6-astra', tools: { getWeather: tool({ description: 'Get the current weather for a location', inputSchema: z.object({ location: z.string().describe('City name, e.g. San Francisco'), }), execute: async ({ location }) => ({ location, temperature: 72, condition: 'sunny', }), }), }, prompt: "What's the weather in Tokyo?", }); return Response.json({ text }); } ``` ### 8\. Keep your agent available with fallbacks An agent makes many model calls per task, and each one is a chance to hit a provider outage or error. That makes failover matter more for agents, not less. Pass a `models` array in `providerOptions.gateway` to list backup models, which the Gateway tries in order when the primary model fails: ```typescript import { createUIMessageStreamResponse, streamText, toUIMessageStream } from 'ai'; export async function POST(request: Request) { const { prompt } = await request.json(); const result = streamText({ model: 'openai/gpt-6-astra', // Primary model prompt, providerOptions: { gateway: { models: ['anthropic/claude-fable-5.1', 'google/gemini-3.8-flash'], // Fallbacks }, }, }); return createUIMessageStreamResponse({ stream: toUIMessageStream({ stream: result.stream }), }); } ``` In this example, the Gateway first attempts the primary model. If that fails, it tries `anthropic/claude-fable-5.1`, then `google/gemini-3.8-flash`. The response comes from the first model that succeeds, and failover happens automatically without changes to your application logic. ## Let your agent run code safely Capable agents often need more than predefined tools. To compute a result, transform data, or test its own output, an agent may generate code and run it. Executing model-generated code on your own infrastructure is risky, because the code might consume excessive resources, read sensitive files, make unwanted network requests, or run destructive commands. [Vercel Sandbox](https://vercel.com/docs/sandbox) provides the agent with an isolated environment to run the code. Each sandbox is an isolated Linux microVM with resource limits and automatic timeouts, so untrusted code runs without touching your production systems. Sandboxes are persistent by default: stopping one automatically snapshots its filesystem so you can resume it later, and each snapshot consumes storage that's billed separately. For one-off code execution like the pattern below, pass `persistent: false` at creation time so no snapshot is created. Add the [Vercel Sandbox SDK](https://vercel.com/docs/sandbox/sdk-reference) to your project: **pnpm** ```bash pnpm i @vercel/sandbox ``` **npm** ```bash npm i @vercel/sandbox ``` **yarn** ```bash yarn add @vercel/sandbox ``` **bun** ```bash bun add @vercel/sandbox ``` The pattern has two parts: generate code with the model via the AI Gateway, then write it to a fresh sandbox and run it. The sandbox is created, used, and stopped within a single request: ````typescript import { generateText } from 'ai'; import { Sandbox } from '@vercel/sandbox'; const SYSTEM_PROMPT = `You are a code generator. Write JavaScript that runs in Node.js. Output only the code, with no explanations or markdown.`; async function generateCode(task: string): Promise { const { text } = await generateText({ model: 'anthropic/claude-fable-5.1', instructions: SYSTEM_PROMPT, prompt: `Write JavaScript code to: ${task}`, }); // Strip markdown fences the model may add despite the instructions return text .replace(/^\s*```(?:javascript|js)?\s*/i, '') .replace(/\s*```\s*$/i, '') .trim(); } async function executeCode(code: string) { const sandbox = await Sandbox.create({ resources: { vcpus: 2 }, timeout: 120_000, // 2 minutes persistent: false, // One-off execution, no filesystem snapshot needed }); try { await sandbox.writeFiles([ { path: '/vercel/sandbox/code.mjs', content: Buffer.from(code) }, ]); const result = await sandbox.runCommand({ cmd: 'node', args: ['code.mjs'] }); const stdout = await result.stdout(); const stderr = await result.stderr(); return { output: stdout || stderr || '(no output)', exitCode: result.exitCode }; } finally { await sandbox.stop(); } } ```` Without an `image` option, the sandbox boots from the Ubuntu-based `vercel/sandbox/universal` image, which includes Node.js, Python, and common tooling. To pin a version, pass an image like `vercel/sandbox/node:24` (the older `runtime` option still works but is deprecated). The system prompt and the sandbox give the agent two layers of safety. The prompt steers the model away from dangerous operations, and the sandbox enforces isolation whatever the model produces. It also captures `stdout` and `stderr`, so the agent can read failures and retry without touching your host. ## Bring your agent to your users The agent you've built runs over an HTTP route, but your users may already be in Slack, Microsoft Teams, Discord, or Google Chat. [Chat SDK](https://chat-sdk.dev/) is a TypeScript library for building chatbots that work across platforms like these from a single codebase, and it integrates directly with AI SDK. That means the same agent you call through AI Gateway can answer inside a thread without rebuilding it for each platform. Two helpers from the `chat/ai` subpath connect the two SDKs. ### Feed thread history to your agent `toAiMessages` converts an array of Chat SDK `Message` objects into the `{ role, content }[]` format the AI SDK expects. The output is compatible with the AI SDK's `ModelMessage[]`, so you can pass it straight into a model call. The example below fetches recent messages from the thread, converts them, and passes the result as the agent's prompt: ```typescript import { toAiMessages } from 'chat/ai'; bot.onSubscribedMessage(async (thread, message) => { const result = await thread.adapter.fetchMessages(thread.id, { limit: 20 }); const history = await toAiMessages(result.messages); const response = await agent.stream({ prompt: history }); await thread.post(response.fullStream); }); ``` `toAiMessages` maps messages authored by the bot to the `assistant` role and all others to `user`, sorts them chronologically, and includes image and text attachments as multipart content. For multi-user threads, pass `includeNames: true` so the agent can tell speakers apart and address them accordingly. ### Let your agent act on the platform `createChatTools` provides the agent with a set of pre-built AI SDK tools for posting messages, adding reactions, and performing other platform actions. Write operations require user approval by default, which pauses the agent until a human responds. For an agent that should act without intervention, pass `requireApproval: false`: ```typescript import { Chat } from 'chat'; import { createChatTools } from 'chat/ai'; import { createSlackAdapter } from '@chat-adapter/slack'; import { createMemoryState } from '@chat-adapter/state-memory'; import { generateText } from 'ai'; const chat = new Chat({ userName: 'mybot', adapters: { slack: createSlackAdapter() }, state: createMemoryState(), }); const result = await generateText({ model: 'google/gemini-3.8-flash', tools: createChatTools({ chat, preset: 'messenger', requireApproval: false, // Write tools pause for approval by default }), prompt: 'Post a friendly hello in slack:C0123ABC and react to it with a thumbs up.', }); ``` Each tool resolves the right adapter from the ID prefix you give it (e.g., `slack:`), so one agent can drive any platform your `Chat` instance is wired up to. ## Give your agent secure access to third-party APIs Once your agent acts on outside services, such as opening GitHub pull requests or querying a data warehouse, it needs credentials. Bundling long-lived API keys into your deployment is risky: the secret sits in your environment indefinitely, applies to every request, and is hard to scope or revoke. [Vercel Connect](https://vercel.com/docs/connect) solves this by issuing short-lived provider tokens at runtime instead. You register a connector for a provider once, link it to your projects and environments, and your code requests a scoped token only when it needs one. Add the [Vercel Connect SDK](https://vercel.com/docs/connect/ts-sdk-reference) to your project: **pnpm** ```bash pnpm i @vercel/connect ``` **npm** ```bash npm i @vercel/connect ``` **yarn** ```bash yarn add @vercel/connect ``` **bun** ```bash bun add @vercel/connect ``` Request a token with `getToken`, passing the connector, a subject, and the scopes you need. The subject controls whose identity the token represents. Pick the variant that matches how your agent should act: - **Act as your app**: `{ type: 'app' }` requests a token that represents your service itself and its associated permissions. - **Act on behalf of a user**: `{ type: 'user', id: '...' }` represents a specific user who authorized access through the connector once. **Act as your app** ```typescript import { getToken } from '@vercel/connect'; const token = await getToken('slack/acme-slack', { subject: { type: 'app' }, scopes: ['chat:write'], }); ``` **Act on behalf of a user** ```typescript import { getToken } from '@vercel/connect'; const token = await getToken('slack/acme-slack', { subject: { type: 'user', id: 'user_abc123' }, scopes: ['chat:write'], }); ``` Whichever subject you choose, the SDK caches tokens in-process and refreshes them automatically, so an agent that makes multiple provider calls in a single run requests one token rather than one per call. Vercel Connect supports 100+ services through preset and managed connectors, plus generic OAuth, API key, and MCP server connectors. To set up your first connector, see [The Complete Guide to Vercel Connect](https://vercel.com/kb/guide/vercel-connect). ## Build durable agents with the Workflow SDK The agents you've built so far run in memory. If the process crashes, the function times out, or the user refreshes the page, the agent's progress is lost. That's fine for short, single-tool interactions, but it's costly for production agents that chain several tool calls, such as booking a flight or running a research task across multiple APIs. `WorkflowAgent` from `@ai-sdk/workflow`, available in AI SDK 7, runs the same agent loop as the standard in-memory agent, but inside a [Vercel Workflow](https://vercel.com/docs/workflows). Each tool call becomes a durable step, so progress persists across process boundaries, and failed steps are retried from the last checkpoint instead of restarting the whole loop. Tools marked `needsApproval` can suspend the agent for hours or days until a user responds, which makes human-in-the-loop flows possible without a custom state store or polling. Here's how the two runtimes compare: | | In-memory agent | `WorkflowAgent` | | ---------------------------------------------------------------- | ---------------------------------- | -------------------------------- | | State | Lost on crash or time out | Persisted after each tool call | | Failure recovery | The whole loop restarts | Retries from the last checkpoint | | Human approval | Custom state store and polling | Built in via `needsApproval` | | [Observability](https://vercel.com/docs/workflows#observability) | Application logs | Steps in the dashboard | | Best for | Short, single-request interactions | Tasks that outlive a request | To get durability, the agent runs inside a function marked `'use workflow'`, and each tool's `execute` function is marked `'use step'`: ```typescript import { WorkflowAgent, type ModelCallStreamPart } from '@ai-sdk/workflow'; import { convertToModelMessages, tool, type UIMessage } from 'ai'; import { getWritable } from 'workflow'; import { z } from 'zod'; async function searchFlightsStep(input: { origin: string; destination: string; date: string; }) { 'use step'; const response = await fetch(`https://api.flights.example/search?...`); return response.json(); } export async function chat(messages: UIMessage[]) { 'use workflow'; const modelMessages = await convertToModelMessages(messages); const agent = new WorkflowAgent({ model: 'openai/gpt-6-astra', instructions: 'You are a flight booking assistant.', tools: { searchFlights: tool({ description: 'Search for available flights', inputSchema: z.object({ origin: z.string(), destination: z.string(), date: z.string(), }), execute: searchFlightsStep, }), }, }); const result = await agent.stream({ messages: modelMessages, writable: getWritable(), }); return { messages: result.messages }; } ``` The model string is the same `creator/model-name` form you've used throughout, so the request still routes through AI Gateway with its failover and unified billing. What changes is the runtime: tool calls now persist, retry, and appear as steps in the [Vercel dashboard](https://vercel.com/d?to=%2F%5Bteam%5D%2F~%2Fworkflows). To add human approval, set `needsApproval: true` on a tool definition, which suspends the durable workflow until the user responds. Start with the standard in-memory agent, and reach for `WorkflowAgent` when tool calls outlive their request, approvals exceed function timeouts, or each call should be independently retryable and traced. ## Best practices #### Refresh your OIDC token during local development OIDC tokens are valid for 12 hours, so a long local session eventually outlasts one. When requests start failing with an authentication error, the token has likely expired. Run `vercel env pull` to write a fresh token to your environment file. Deployed environments provision tokens automatically, so this only applies locally. #### Set fallbacks for production traffic A single model and provider is a single point of failure, and an agent makes many model calls per task, so the exposure compounds. List two or three fallback models in `providerOptions.gateway`. The Gateway tries them in order, keeping requests available when one provider has an outage. #### Keep model IDs in configuration Switching models is a single string change, so store model IDs in environment variables or a config file rather than hardcoding them. This lets you switch providers without editing application code, and makes it straightforward to adopt newer models as they land on the Gateway. #### Confirm your AI SDK version This guide targets AI SDK 7, which requires Node.js 22 or later and provides `WorkflowAgent`. Run `pnpm list ai` to check your installed version, and see the [AI SDK migration guides](https://ai-sdk.dev/docs/migration-guides) before upgrading from an earlier major version. #### Opt out of Sandbox persistence for one-off runs Sandboxes automatically snapshot their filesystem on stop by default, and each snapshot is billed as separate snapshot storage. Pass `persistent: false` at creation time when you won't resume the sandbox, so one-off runs don't accrue storage for state they never reuse. ## Resources and next steps - Learn about [model routing and fallbacks](https://vercel.com/docs/ai-gateway/models-and-providers/provider-options) for finer provider control - Read more about [OIDC authentication](https://vercel.com/docs/ai-gateway/authentication-and-byok/authentication) and how tokens work on Vercel - Explore the [AI SDK documentation](https://ai-sdk.dev/getting-started) for advanced patterns - Run AI-generated code safely with [Vercel Sandbox](https://vercel.com/docs/sandbox) and the [code execution guide](https://vercel.com/kb/guide/how-to-execute-ai-generated-code-safely) - Build a cross-platform chatbot with [Chat SDK](https://chat-sdk.dev/docs) and its [AI SDK integration](https://chat-sdk.dev/docs/ai/ai-sdk-tools) - Request short-lived provider tokens at runtime with [Vercel Connect](https://vercel.com/docs/connect) - Make your agents durable with [WorkflowAgent](https://vercel.com/kb/guide/what-is-workflowagent) and the [Workflow SDK](https://workflow-sdk.dev/) - Browse the [model library](https://vercel.com/ai-gateway/models) to see every supported provider and model ## Related Resources - [Software factory](https://vercel.com/templates/eve/eve-software-factory): Software factory built on eve: AI agents work each stage of the development loop, and people make the judgment calls. - [Incident response](https://vercel.com/templates/eve/eve-incident-response-agent): sre investigates production issues using a hypothesis-driven approach and outputs verifiable evidence from sources. - [Personal agent](https://vercel.com/templates/nuxt/eve-personal-agent): A durable AI assistant with long-term memory. Chat on the web or Slack, query Linear, and pick up where you left off.