Using TanStack AI with Vercel AI Gateway

Managing a separate API key, SDK, and billing account for every model provider adds friction as your application grows. Vercel AI Gateway solves this with a single OpenAI-compatible endpoint and one API key that reach hundreds of models, with provider routing, fallbacks, and spend monitoring built in. TanStack AI connects to the Gateway through its official adapter, @tanstack/ai-vercel-gateway, so you can pick a provider per request without changing your application code.

Copy link to headingOverview

In this guide, you'll learn how to:

• Install and authenticate the @tanstack/ai-vercel-gateway adapter
• Stream chat responses through AI Gateway with the useChat hook
• Configure provider routing and fallbacks per request
• Generate embeddings and images with the same API key

Copy link to headingPrerequisites

Before you begin, make sure you have:

• A Vercel account
• A TanStack AI project with @tanstack/ai and a framework package such as @tanstack/ai-react installed

For local development, you also need Node.js 22+ and the Vercel CLI.

Copy link to headingHow it works

The adapter talks to AI Gateway's public OpenAI-compatible API at https://ai-gateway.vercel.sh/v1. By default it uses the OpenAI Responses API, and model ids follow the creator/model format, such as openai/gpt-6-astra.

The adapter's model catalog is a closed list generated from the Gateway's GET /v1/models endpoint and updated daily. Each model carries its own typed modelOptions and input types, so a text-only model rejects image parts at the type level, and a model without temperature in the catalog doesn't accept a temperature option.

Copy link to headingSteps

Copy link to heading1. Install the adapter

Install the package with your package manager:

pnpm i @tanstack/ai-vercel-gateway

npm i @tanstack/ai-vercel-gateway

yarn add @tanstack/ai-vercel-gateway

bun add @tanstack/ai-vercel-gateway

Copy link to heading2. Create an API key

The adapter reads AI_GATEWAY_API_KEY from your environment automatically. If that variable isn't set, it falls back to VERCEL_OIDC_TOKEN.

To create an API key:

1. Go to the AI Gateway tab in your Vercel dashboard.
2. Select API Keys in the sidebar.
3. Click Create API Key and follow the steps.
4. Save the key to your environment file:

AI_GATEWAY_API_KEY=your_api_key_here

If your application runs on Vercel, you can skip API keys entirely. Vercel generates an OIDC token for your project, and the adapter can use that too.

To use the token locally, run vercel link and vercel env pull to download it. OIDC tokens expire after 12 hours, so run vercel env pull again.

You can also pass a key explicitly to a create* factory instead of relying on the adapter's automatic lookup. This is useful when the key comes from a runtime credential service rather than an environment variable.

For example, store your AI Gateway API key in a Vercel Connect API key connector:

vercel connect create aig-api --name acme-ai-gateway

Then request the key with getToken when you construct the adapter:

import { getToken } from "@vercel/connect"
import { createVercelGatewayText } from "@tanstack/ai-vercel-gateway"

const token = await getToken("api-key/acme-ai-gateway")

const adapter = createVercelGatewayText("anthropic/claude-opus-5", token)

Vercel Connect stores the key once and returns it to your server-side code at runtime, so no long-lived secret lives in your application config.

Each getToken call is also logged in the connector's Observability tab with the requesting project and environment, giving you an audit trail of where your Gateway key is used. See Connect observability for event types and retention.

Copy link to heading3. Create a chat endpoint

On the server, pass the adapter to TanStack AI's chat function and return the stream as server-sent events (SSE):

import { chat, toServerSentEventsResponse } from "@tanstack/ai"
import { vercelGatewayText } from "@tanstack/ai-vercel-gateway"

export async function POST(request: Request) {
  const { messages } = await request.json()

  const stream = chat({
    adapter: vercelGatewayText("anthropic/claude-opus-5"),
    messages,
  })

  return toServerSentEventsResponse(stream)
}

The adapter is the only Gateway-specific piece. Swapping to a different provider means changing the model id string, not the endpoint code.

Copy link to heading4. Connect the client

On the client, the useChat hook works the same as with every other provider:

import { useState } from "react"
import { fetchServerSentEvents, useChat } from "@tanstack/ai-react"

export function Chat() {
  const [input, setInput] = useState("")

  const { messages, sendMessage, isLoading } = useChat({
    connection: fetchServerSentEvents("/api/chat"),
  })

  return (
    <div>
      {messages.map((message) => (
        <div key={message.id}>
          <strong>{message.role}</strong>
          {message.parts.map((part, index) =>
            part.type === "text" ? <p key={index}>{part.content}</p> : null,
          )}
        </div>
      ))}

      <form
        onSubmit={(event) => {
          event.preventDefault()
          if (!input.trim() || isLoading) return
          sendMessage(input)
          setInput("")
        }}
      >
        <input value={input} onChange={(event) => setInput(event.target.value)} />
        <button type="submit" disabled={isLoading}>
          Send
        </button>
      </form>
    </div>
  )
}

Copy link to heading5. Switch APIs when a model requires it

The adapter defaults to the OpenAI Responses API. If a model must use Chat Completions instead, pass { api: "chat" } as the second argument:

import { chat } from "@tanstack/ai"
import { vercelGatewayText } from "@tanstack/ai-vercel-gateway"

const stream = chat({
  adapter: vercelGatewayText("openai/gpt-5.5", { api: "chat" }),
  messages: [{ role: "user", content: "Hello" }],
})

api: "responses" matches the default behavior, and api: "chat-completions" is an alias for api: "chat".

Copy link to headingConfigure provider routing and fallbacks

AI Gateway can try providers in a preferred order, restrict which providers run, and fall back to alternate models when a request fails.

Set these routing options on modelOptions.gateway, and the adapter sends them to the Gateway as providerOptions.gateway. Don't place gateway at the top level of the request body.

import { chat } from "@tanstack/ai"
import { vercelGatewayText } from "@tanstack/ai-vercel-gateway"

const stream = chat({
  adapter: vercelGatewayText("anthropic/claude-opus-5"),
  messages: [{ role: "user", content: "Hello" }],
  modelOptions: {
    gateway: {
      order: ["anthropic", "openai"],
      only: ["anthropic"],
      sort: "cost",
      models: ["anthropic/claude-opus-5", "openai/gpt-5.5"],
      caching: "auto",
      disallowPromptTraining: true,
    },
  },
})

order and only accept catalog provider ids such as "anthropic", and models accepts catalog chat model ids.

Copy link to headingGenerate embeddings

Use vercelGatewayEmbedding with TanStack AI's embed function to create embeddings through the same API key:

import { embed } from "@tanstack/ai"
import { vercelGatewayEmbedding } from "@tanstack/ai-vercel-gateway"

const result = await embed({
  adapter: vercelGatewayEmbedding("openai/text-embedding-3-small"),
  input: "a red guitar",
})

console.log(result.embeddings[0]?.vector)

Copy link to headingGenerate images

Use vercelGatewayImage for text-to-image generation. The adapter calls the Gateway's POST /v1/images/generations endpoint:

import { generateImage } from "@tanstack/ai"
import { vercelGatewayImage } from "@tanstack/ai-vercel-gateway"

const result = await generateImage({
  adapter: vercelGatewayImage("openai/gpt-image-1"),
  prompt: "a red guitar",
})

Copy link to headingSummarize text

Use vercelGatewaySummarize with the summarize function:

import { summarize } from "@tanstack/ai"
import { vercelGatewaySummarize } from "@tanstack/ai-vercel-gateway"

const result = await summarize({
  adapter: vercelGatewaySummarize("anthropic/claude-opus-5"),
  text: "The Fender Stratocaster is a versatile electric guitar.",
  stream: false,
})

Copy link to headingLimitations

The @tanstack/ai-vercel-gateway package covers chat, embeddings, text-to-image generation, and summarization. It doesn't generate video or handle speech, transcription, or reranking.

For those tasks, install a different adapter package built for that capability:

Copy link to headingNext steps

• Learn about routing, fallbacks, and budgets in the AI Gateway documentation
• Review AI Gateway authentication options, including OIDC tokens and BYOK
• Read how Vercel Connect tokens issue short-lived credentials at runtime
• Explore the full adapter reference in the TanStack AI Vercel AI Gateway docs
• Run coding agents in isolated microVMs with Vercel Sandbox
• Build a Jev agent with TanStack Start and TanStack AI
• Use the adapter package's vercelGatewayDecider with Jev in Moderate product reviews with Jev, TanStack AI, and AI Gateway to get typed choices, scores, and boolean probabilities instead of generated text