Build AI agents with AI Gateway and AI SDK

Building an AI agent usually means managing a separate account, API key, and rate limit for every model provider you want to try. With the AI SDK and AI Gateway, a single endpoint and set of credentials can access hundreds of models, and switching providers is a one-string change.

Once the agent works, Vercel Sandbox runs the code it writes in isolated microVMs, Chat SDK brings it to dozens of platforms, Vercel Connect gives it scoped access to third-party APIs through short-lived tokens, and the Workflow SDK keeps it running through crashes and timeouts.

Copy link to headingOverview

In this guide, you'll learn how to:

• Set up a Next.js project and authenticate to AI Gateway with OIDC tokens
• Generate text, stream responses, and produce structured outputs
• Give your agent tools so they can act, not just respond
• Keep your agent available with model fallbacks
• Run AI-generated code safely in isolated Vercel Sandbox microVMs
• Bring your agent to Slack, Teams, and other chat platforms with Chat SDK
• Give your agent scoped access to third-party APIs with Vercel Connect
• Make your agent durable and resumable with the Workflow SDK

Copy link to headingPrerequisites

Before you begin, make sure you have:

• A Vercel account.
• Vercel CLI installed (npm i -g vercel)
• Node.js 22+ and a package manager (e.g., pnpm)

Copy link to headingHow it works

AI Gateway is a single endpoint that sits in front of every supported provider.

You send it a model string in the form creator/model-name, and the Gateway resolves the provider, authenticates, routes the request, and tracks usage. AI Gateway is the default provider in AI SDK 7, so a plain model string routes through it with no configuration, and your application code stays the same whether you call GPT-6 Astra, Claude Fable 5.1, or Gemini 3.8 Flash. Tokens cost the same as they would from the provider directly, with no markup.

The AI SDK provides the function-level API you'll build the agent from, and AI Gateway provides the infrastructure underneath: authentication, usage tracking, failover, and billing. The two have high cohesion but loose coupling, so you can adopt the SDK on its own and add Gateway features via provider options.

You'll use three core AI SDK functions in this guide:

Copy link to headingSteps

Copy link to heading1. Create a Next.js app

Use create-next-app to bootstrap a new project:

pnpm create next-app@latest ai-gateway-demo --yes

npx create-next-app@latest ai-gateway-demo --yes

yarn create next-app@latest ai-gateway-demo --yes

bun create next-app@latest ai-gateway-demo --yes

The --yes flag uses the recommended defaults: TypeScript, Tailwind CSS, ESLint, App Router, and Turbopack, with the @/* import alias. Omit the flag if you want to customize these options interactively.

Copy link to heading2. Install the AI SDK

Add the ai package to your project:

pnpm i ai

npm i ai

yarn add ai

bun add ai

This installs AI SDK 7, which uses AI Gateway as its default provider.

If you're on an earlier major version, review the AI SDK migration guides before upgrading, and note that the WorkflowAgent used later in this guide requires v7.

Copy link to heading3. Authenticate with OIDC

AI Gateway authenticates requests using a Vercel OIDC token, which is automatically generated for your project.

First, link your local directory to a Vercel project:

vercel link

Then pull the development environment variables:

vercel env pull

This writes the variables to your local environment file. The OIDC token will be included, and it will be valid for 12 hours. You'll need to run vercel env pull again to refresh the token when it expires.

Copy link to heading4. Generate text

Start with the simplest request: generate a single block of text.

Create an API route at app/api/chat/route.ts. Pass a plain string model ID to generateText and the AI Gateway resolves the provider and routes the request. With OIDC authentication, you don't reference a key anywhere in your code:

import { generateText } from 'ai';

export async function GET() {
  const { text } = await generateText({
    model: 'openai/gpt-6-astra',
    prompt: 'Explain quantum computing in one paragraph.',
  });

  return Response.json({ text });
}

Start the dev server and visit the route to see the response:

pnpm dev

Copy link to heading5. Stream responses

For real-time output, use streamText and return a streamed response. This is the pattern you'll use for chat interfaces and any response long enough that waiting for the full result would hurt the experience:

import { createUIMessageStreamResponse, streamText, toUIMessageStream } from 'ai';

export async function POST(request: Request) {
  const { prompt } = await request.json();

  const result = streamText({
    model: 'anthropic/claude-fable-5.1',
    prompt,
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({ stream: result.stream }),
  });
}

Copy link to heading6. Generate structured outputs

Use generateObject with a Zod schema to get type-safe structured data:

import { generateObject } from 'ai';
import { z } from 'zod';

export async function GET() {
  const { object } = await generateObject({
    model: 'google/gemini-3.8-flash',
    schema: z.object({
      name: z.string(),
      age: z.number(),
      city: z.string(),
    }),
    prompt: 'Extract: John is 30 years old and lives in NYC.',
  });

  return Response.json(object); // { name: 'John', age: 30, city: 'NYC' }
}

Copy link to heading7. Give your agent tools

So far, the model has produced text and data, but it hasn't done anything.

Tools change that. You define functions that the model can invoke to fetch data, call an API, or act on the outside world. The AI SDK then runs the model in a loop, calling your tools and feeding each result back until the task is done.

Define a tool with a description, an input schema, and an execute function:

import { generateText, tool } from 'ai';
import { z } from 'zod';

export async function GET() {

  const { text } = await generateText({
    model: 'openai/gpt-6-astra',
    tools: {
      getWeather: tool({
        description: 'Get the current weather for a location',
        inputSchema: z.object({
          location: z.string().describe('City name, e.g. San Francisco'),
        }),
        execute: async ({ location }) => ({
          location,
          temperature: 72,
          condition: 'sunny',
        }),
      }),
    },
    prompt: "What's the weather in Tokyo?",
  });

  return Response.json({ text });
}

Copy link to heading8. Keep your agent available with fallbacks

An agent makes many model calls per task, and each one is a chance to hit a provider outage or error. That makes failover matter more for agents, not less. Pass a models array in providerOptions.gateway to list backup models, which the Gateway tries in order when the primary model fails:

import { createUIMessageStreamResponse, streamText, toUIMessageStream } from 'ai';

export async function POST(request: Request) {
  const { prompt } = await request.json();

  const result = streamText({
    model: 'openai/gpt-6-astra', // Primary model
    prompt,
    providerOptions: {
      gateway: {
        models: ['anthropic/claude-fable-5.1', 'google/gemini-3.8-flash'], // Fallbacks
      },
    },
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({ stream: result.stream }),
  });
}

In this example, the Gateway first attempts the primary model. If that fails, it tries anthropic/claude-fable-5.1, then google/gemini-3.8-flash. The response comes from the first model that succeeds, and failover happens automatically without changes to your application logic.

Copy link to headingLet your agent run code safely

Capable agents often need more than predefined tools. To compute a result, transform data, or test its own output, an agent may generate code and run it. Executing model-generated code on your own infrastructure is risky, because the code might consume excessive resources, read sensitive files, make unwanted network requests, or run destructive commands.

Vercel Sandbox provides the agent with an isolated environment to run the code. Each sandbox is an isolated Linux microVM with resource limits and automatic timeouts, so untrusted code runs without touching your production systems.

Sandboxes are persistent by default: stopping one automatically snapshots its filesystem so you can resume it later, and each snapshot consumes storage that's billed separately. For one-off code execution like the pattern below, pass persistent: false at creation time so no snapshot is created.

Add the Vercel Sandbox SDK to your project:

pnpm i @vercel/sandbox

npm i @vercel/sandbox

yarn add @vercel/sandbox

bun add @vercel/sandbox

The pattern has two parts: generate code with the model via the AI Gateway, then write it to a fresh sandbox and run it.

The sandbox is created, used, and stopped within a single request:

import { generateText } from 'ai';
import { Sandbox } from '@vercel/sandbox';

const SYSTEM_PROMPT = `You are a code generator. Write JavaScript that runs in Node.js.
Output only the code, with no explanations or markdown.`;

async function generateCode(task: string): Promise<string> {
  const { text } = await generateText({
    model: 'anthropic/claude-fable-5.1',
    instructions: SYSTEM_PROMPT,
    prompt: `Write JavaScript code to: ${task}`,
  });

  // Strip markdown fences the model may add despite the instructions
  return text
    .replace(/^\s*```(?:javascript|js)?\s*/i, '')
    .replace(/\s*```\s*$/i, '')
    .trim();
}

async function executeCode(code: string) {
  const sandbox = await Sandbox.create({
    resources: { vcpus: 2 },
    timeout: 120_000, // 2 minutes
    persistent: false, // One-off execution, no filesystem snapshot needed
  });

  try {
    await sandbox.writeFiles([
      { path: '/vercel/sandbox/code.mjs', content: Buffer.from(code) },
    ]);

    const result = await sandbox.runCommand({ cmd: 'node', args: ['code.mjs'] });
    const stdout = await result.stdout();
    const stderr = await result.stderr();

    return { output: stdout || stderr || '(no output)', exitCode: result.exitCode };
  } finally {
    await sandbox.stop();
  }
}

Without an image option, the sandbox boots from the Ubuntu-based vercel/sandbox/universal image, which includes Node.js, Python, and common tooling. To pin a version, pass an image like vercel/sandbox/node:24 (the older runtime option still works but is deprecated).

The system prompt and the sandbox give the agent two layers of safety. The prompt steers the model away from dangerous operations, and the sandbox enforces isolation whatever the model produces. It also captures stdout and stderr, so the agent can read failures and retry without touching your host.

Copy link to headingBring your agent to your users

The agent you've built runs over an HTTP route, but your users may already be in Slack, Microsoft Teams, Discord, or Google Chat. Chat SDK is a TypeScript library for building chatbots that work across platforms like these from a single codebase, and it integrates directly with AI SDK. That means the same agent you call through AI Gateway can answer inside a thread without rebuilding it for each platform.

Two helpers from the chat/ai subpath connect the two SDKs.

Copy link to headingFeed thread history to your agent

toAiMessages converts an array of Chat SDK Message objects into the { role, content }[] format the AI SDK expects. The output is compatible with the AI SDK's ModelMessage[], so you can pass it straight into a model call.

The example below fetches recent messages from the thread, converts them, and passes the result as the agent's prompt:

import { toAiMessages } from 'chat/ai';

bot.onSubscribedMessage(async (thread, message) => {
  const result = await thread.adapter.fetchMessages(thread.id, { limit: 20 });
  const history = await toAiMessages(result.messages);
  const response = await agent.stream({ prompt: history });
  await thread.post(response.fullStream);
});

toAiMessages maps messages authored by the bot to the assistant role and all others to user, sorts them chronologically, and includes image and text attachments as multipart content.

For multi-user threads, pass includeNames: true so the agent can tell speakers apart and address them accordingly.

Copy link to headingLet your agent act on the platform

createChatTools provides the agent with a set of pre-built AI SDK tools for posting messages, adding reactions, and performing other platform actions. Write operations require user approval by default, which pauses the agent until a human responds.

For an agent that should act without intervention, pass requireApproval: false:

import { Chat } from 'chat';
import { createChatTools } from 'chat/ai';
import { createSlackAdapter } from '@chat-adapter/slack';
import { createMemoryState } from '@chat-adapter/state-memory';
import { generateText } from 'ai';

const chat = new Chat({
  userName: 'mybot',
  adapters: { slack: createSlackAdapter() },
  state: createMemoryState(),
});

const result = await generateText({
  model: 'google/gemini-3.8-flash',
  tools: createChatTools({
    chat,
    preset: 'messenger',
    requireApproval: false, // Write tools pause for approval by default
  }),
  prompt: 'Post a friendly hello in slack:C0123ABC and react to it with a thumbs up.',
});

Each tool resolves the right adapter from the ID prefix you give it (e.g., slack:), so one agent can drive any platform your Chat instance is wired up to.

Copy link to headingGive your agent secure access to third-party APIs

Once your agent acts on outside services, such as opening GitHub pull requests or querying a data warehouse, it needs credentials. Bundling long-lived API keys into your deployment is risky: the secret sits in your environment indefinitely, applies to every request, and is hard to scope or revoke.

Vercel Connect solves this by issuing short-lived provider tokens at runtime instead. You register a connector for a provider once, link it to your projects and environments, and your code requests a scoped token only when it needs one.

Add the Vercel Connect SDK to your project:

pnpm i @vercel/connect

npm i @vercel/connect

yarn add @vercel/connect

bun add @vercel/connect

Request a token with getToken, passing the connector, a subject, and the scopes you need. The subject controls whose identity the token represents.

Pick the variant that matches how your agent should act:

• Act as your app: { type: 'app' } requests a token that represents your service itself and its associated permissions.
• Act on behalf of a user: { type: 'user', id: '...' } represents a specific user who authorized access through the connector once.

import { getToken } from '@vercel/connect';

const token = await getToken('slack/acme-slack', {
  subject: { type: 'app' },
  scopes: ['chat:write'],
});

Whichever subject you choose, the SDK caches tokens in-process and refreshes them automatically, so an agent that makes multiple provider calls in a single run requests one token rather than one per call.

Vercel Connect supports 100+ services through preset and managed connectors, plus generic OAuth, API key, and MCP server connectors.

To set up your first connector, see The Complete Guide to Vercel Connect.

Copy link to headingBuild durable agents with the Workflow SDK

The agents you've built so far run in memory. If the process crashes, the function times out, or the user refreshes the page, the agent's progress is lost.

That's fine for short, single-tool interactions, but it's costly for production agents that chain several tool calls, such as booking a flight or running a research task across multiple APIs.

WorkflowAgent from @ai-sdk/workflow, available in AI SDK 7, runs the same agent loop as the standard in-memory agent, but inside a Vercel Workflow.

Each tool call becomes a durable step, so progress persists across process boundaries, and failed steps are retried from the last checkpoint instead of restarting the whole loop. Tools marked needsApproval can suspend the agent for hours or days until a user responds, which makes human-in-the-loop flows possible without a custom state store or polling.

Here's how the two runtimes compare:

To get durability, the agent runs inside a function marked 'use workflow', and each tool's execute function is marked 'use step':

import { WorkflowAgent, type ModelCallStreamPart } from '@ai-sdk/workflow';
import { convertToModelMessages, tool, type UIMessage } from 'ai';
import { getWritable } from 'workflow';
import { z } from 'zod';

async function searchFlightsStep(input: {
  origin: string;
  destination: string;
  date: string;
}) {
  'use step';
  const response = await fetch(`https://api.flights.example/search?...`);
  return response.json();
}

export async function chat(messages: UIMessage[]) {
  'use workflow';
  const modelMessages = await convertToModelMessages(messages);

  const agent = new WorkflowAgent({
    model: 'openai/gpt-6-astra',
    instructions: 'You are a flight booking assistant.',
    tools: {
      searchFlights: tool({
        description: 'Search for available flights',
        inputSchema: z.object({
          origin: z.string(),
          destination: z.string(),
          date: z.string(),
        }),
        execute: searchFlightsStep,
      }),
    },
  });

  const result = await agent.stream({
    messages: modelMessages,
    writable: getWritable<ModelCallStreamPart>(),
  });

  return { messages: result.messages };
}

The model string is the same creator/model-name form you've used throughout, so the request still routes through AI Gateway with its failover and unified billing.

What changes is the runtime: tool calls now persist, retry, and appear as steps in the Vercel dashboard. To add human approval, set needsApproval: true on a tool definition, which suspends the durable workflow until the user responds.

Start with the standard in-memory agent, and reach for WorkflowAgent when tool calls outlive their request, approvals exceed function timeouts, or each call should be independently retryable and traced.

Copy link to headingBest practices

Copy link to headingRefresh your OIDC token during local development

OIDC tokens are valid for 12 hours, so a long local session eventually outlasts one. When requests start failing with an authentication error, the token has likely expired.

Run vercel env pull to write a fresh token to your environment file. Deployed environments provision tokens automatically, so this only applies locally.

Copy link to headingSet fallbacks for production traffic

A single model and provider is a single point of failure, and an agent makes many model calls per task, so the exposure compounds.

List two or three fallback models in providerOptions.gateway. The Gateway tries them in order, keeping requests available when one provider has an outage.

Copy link to headingKeep model IDs in configuration

Switching models is a single string change, so store model IDs in environment variables or a config file rather than hardcoding them.

This lets you switch providers without editing application code, and makes it straightforward to adopt newer models as they land on the Gateway.

Copy link to headingConfirm your AI SDK version

This guide targets AI SDK 7, which requires Node.js 22 or later and provides WorkflowAgent.

Run pnpm list ai to check your installed version, and see the AI SDK migration guides before upgrading from an earlier major version.

Copy link to headingOpt out of Sandbox persistence for one-off runs

Sandboxes automatically snapshot their filesystem on stop by default, and each snapshot is billed as separate snapshot storage.

Pass persistent: false at creation time when you won't resume the sandbox, so one-off runs don't accrue storage for state they never reuse.

Copy link to headingResources and next steps

• Learn about model routing and fallbacks for finer provider control
• Read more about OIDC authentication and how tokens work on Vercel
• Explore the AI SDK documentation for advanced patterns
• Run AI-generated code safely with Vercel Sandbox and the code execution guide
• Build a cross-platform chatbot with Chat SDK and its AI SDK integration
• Request short-lived provider tokens at runtime with Vercel Connect
• Make your agents durable with WorkflowAgent and the Workflow SDK
• Browse the model library to see every supported provider and model