Provider Options

AI Gateway Provider Routing and Fallbacks

AI Gateway can route your AI model requests across multiple AI providers. Each provider offers different models, pricing, and performance characteristics. By default, Vercel AI Gateway dynamically chooses the default providers to give you the best experience based on a combination of recent uptime and latency.

Use providerOptions.gateway to control provider order and fallback behavior.

If you want to customize individual AI model provider settings rather than general AI Gateway behavior, please refer to the model-specific provider options in the AI SDK documentation.

Provider filtering, ordering, and sorting

You can use order, only, and sort in providerOptions.gateway to control which providers handle your requests, in what order, and how they are ranked.

providerOptions: {
  gateway: {
    order: ['bedrock', 'anthropic'], // Try Bedrock first, then Anthropic
    only: ['bedrock', 'anthropic'],  // Only allow these two providers
  },
},

You can also use sort to rank providers by a performance or cost metric. The gateway sorts providers by the chosen metric and tries them in that order:

providerOptions: {
  gateway: {
    sort: 'cost', // Sort by cost, latency ('ttft'), or throughput ('tps')
  },
},

For full details, examples, and provider metadata output, see Provider Filtering, Ordering & Sorting.

Automatic caching

You can use caching: 'auto' in providerOptions.gateway to let AI Gateway automatically apply the appropriate caching strategy based on the provider. This is useful for providers like Anthropic and MiniMax that require explicit cache markers.

providerOptions: {
  gateway: {
    caching: 'auto',
  },
},

For full details, supported providers, and examples across all APIs, see Automatic Caching.

Clients on the Responses API can also set these top-level request fields:

cache_anchor_items pins a cache marker at a known-stable prefix position. See Cache anchor.
cache_ttl selects a five-minute or one-hour cache lifetime. See Cache lifetime.

These fields belong at the top level of the Responses API request, not inside providerOptions.gateway.

Provider timeouts

You can set per-provider timeouts to trigger fast failover when a provider is slow to respond. See the dedicated Provider Timeouts documentation.

Model fallbacks

String model fallbacks try backup models when the primary model fails or is unavailable. See Model Fallbacks for failure-based routing.

Decision requests can also add one conditional object to providerOptions.gateway.models. It reruns a successful decision when a confidence, Boolean probability, or compound condition matches. AI Gateway rejects a conditional object with a 400 error on any other request type, and when the conditional object names the primary model itself. See Decision Fallbacks for the complete wire contract and provider-option behavior.

Advanced configuration

Combining AI Gateway provider options with provider-specific options

You can combine AI Gateway provider options with provider-specific options. This allows you to control both the routing behavior and provider-specific settings in the same request:

These examples use AI SDK 7 and the AI SDK for Python beta. Set AI_GATEWAY_API_KEY before running them. See API format differences for setup, request fields, and response handling.

See the AI SDK Gateway provider-options reference for SDK configuration and usage.

provider-options.ts

import { generateText } from 'ai';
 
const { text } = await generateText({
  model: 'anthropic/claude-sonnet-5',
  prompt: 'Explain quantum computing in two sentences.',
  providerOptions: {
    gateway: {
      order: ['anthropic', 'bedrock'],
    },
    anthropic: {
      thinking: {
        type: 'adaptive',
      },
    },
  },
});
 
console.log(text);

provider-options_ai.py

import asyncio
import ai
 
async def main():
    model = ai.get_model("anthropic/claude-sonnet-5")
    messages = [ai.user_message("Explain quantum computing in two sentences.")]
    params = ai.InferenceRequestParams(
        extra_body={"providerOptions": {"gateway": {"order": ["anthropic", "bedrock"]}, "anthropic": {"thinking": {"type": "adaptive"}}}}
    )
    async with ai.stream(model, messages, params=params) as stream:
        async for event in stream:
            if isinstance(event, ai.events.TextDelta):
                print(event.chunk, end="", flush=True)
    print()
 
asyncio.run(main())

provider-options-chat.ts

import OpenAI from 'openai';
 
const client = new OpenAI({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh/v1',
});
 
const response = await client.chat.completions.create({
  model: 'anthropic/claude-sonnet-5',
  messages: [
    {
      role: 'user',
      content: 'Explain quantum computing in two sentences.',
    },
  ],
  // AI Gateway extension fields are not included in the upstream SDK types.
  ...{
    providerOptions: {
      gateway: {
        order: ['anthropic', 'bedrock'],
      },
      anthropic: {
        thinking: {
          type: 'adaptive',
        },
      },
    },
  },
});
 
console.log(response.choices[0]?.message.content);

provider-options_chat.py

import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.environ["AI_GATEWAY_API_KEY"],
    base_url="https://ai-gateway.vercel.sh/v1",
)
 
response = client.chat.completions.create(
    model="anthropic/claude-sonnet-5",
    messages=[{"role": "user", "content": "Explain quantum computing in two sentences."}],
    extra_body={"providerOptions": {"gateway": {"order": ["anthropic", "bedrock"]}, "anthropic": {"thinking": {"type": "adaptive"}}}},
)
 
print(response.choices[0].message.content)

provider-options-chat.sh

curl --fail-with-body https://ai-gateway.vercel.sh/v1/chat/completions \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "anthropic/claude-sonnet-5",
  "messages": [
    {
      "role": "user",
      "content": "Explain quantum computing in two sentences."
    }
  ],
  "providerOptions": {
    "anthropic": {"thinking": {"type": "adaptive"}},
    "gateway": {
      "order": [
        "anthropic",
        "bedrock"
      ]
    }
  }
}'

provider-options-messages.ts

import Anthropic from '@anthropic-ai/sdk';
 
const client = new Anthropic({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh',
});
 
const response = await client.messages.create({
  model: 'anthropic/claude-sonnet-5',
  messages: [
    {
      role: 'user',
      content: 'Explain quantum computing in two sentences.',
    },
  ],
  max_tokens: 1024,
  ...{
    providerOptions: {
      gateway: {
        order: ['anthropic', 'bedrock'],
      },
      anthropic: {
        thinking: {
          type: 'adaptive',
        },
      },
    },
  },
});
 
for (const block of response.content) {
  if (block.type === 'text') console.log(block.text);
}

provider-options_messages.py

import os
from anthropic import Anthropic
 
client = Anthropic(
    api_key=os.environ["AI_GATEWAY_API_KEY"],
    base_url="https://ai-gateway.vercel.sh",
)
 
response = client.messages.create(
    model="anthropic/claude-sonnet-5",
    messages=[{"role": "user", "content": "Explain quantum computing in two sentences."}],
    max_tokens=1024,
    extra_body={"providerOptions": {"gateway": {"order": ["anthropic", "bedrock"]}, "anthropic": {"thinking": {"type": "adaptive"}}}},
)
 
for block in response.content:
    if block.type == "text":
        print(block.text)

provider-options-messages.sh

curl --fail-with-body https://ai-gateway.vercel.sh/v1/messages \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
  "model": "anthropic/claude-sonnet-5",
  "messages": [
    {
      "role": "user",
      "content": "Explain quantum computing in two sentences."
    }
  ],
  "max_tokens": 1024,
  "providerOptions": {
    "anthropic": {"thinking": {"type": "adaptive"}},
    "gateway": {
      "order": [
        "anthropic",
        "bedrock"
      ]
    }
  }
}'

provider-options-responses.ts

import OpenAI from 'openai';
 
const client = new OpenAI({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh/v1',
});
 
const response = await client.responses.create({
  model: 'anthropic/claude-sonnet-5',
  input: 'Explain quantum computing in two sentences.',
  ...{
    providerOptions: {
      gateway: {
        order: ['anthropic', 'bedrock'],
      },
      anthropic: {
        thinking: {
          type: 'adaptive',
        },
      },
    },
  },
});
 
console.log(response.output_text);

provider-options_responses.py

import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.environ["AI_GATEWAY_API_KEY"],
    base_url="https://ai-gateway.vercel.sh/v1",
)
 
response = client.responses.create(
    model="anthropic/claude-sonnet-5",
    input="Explain quantum computing in two sentences.",
    extra_body={"providerOptions": {"gateway": {"order": ["anthropic", "bedrock"]}, "anthropic": {"thinking": {"type": "adaptive"}}}},
)
 
print(response.output_text)

provider-options-responses.sh

curl --fail-with-body https://ai-gateway.vercel.sh/v1/responses \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "anthropic/claude-sonnet-5",
  "input": "Explain quantum computing in two sentences.",
  "providerOptions": {
    "anthropic": {"thinking": {"type": "adaptive"}},
    "gateway": {
      "order": [
        "anthropic",
        "bedrock"
      ]
    }
  }
}'

The gateway.order option selects the provider sequence. The anthropic.thinking option requests adaptive thinking from Claude Sonnet 5. See Anthropic reasoning for supported models and API format differences for extension fields.

Request-scoped BYOK

You can pass your own provider credentials on a per-request basis using the byok option in providerOptions.gateway. This allows you to use your existing provider accounts for specific requests without configuring credentials in the dashboard.

app/api/chat/route.ts

import { streamText } from 'ai';
 
export async function POST(request: Request) {
  const { prompt } = await request.json();
 
  const result = streamText({
    model: 'anthropic/claude-opus-5',
    prompt,
    providerOptions: {
      gateway: {
        byok: {
          anthropic: [{ apiKey: process.env.ANTHROPIC_API_KEY }],
        },
      },
    },
  });
 
  return result.toUIMessageStreamResponse();
}

For detailed information about credential structures, multiple credentials, and usage with the Chat Completions API, see the BYOK documentation.

Reasoning

For models that support reasoning (also known as "thinking"), you can use providerOptions to configure reasoning behavior. The example below shows how to request high reasoning effort and a detailed summary for OpenAI's gpt-oss-120b model. Providers can differ in which options they honor; setting reasoningSummary doesn't guarantee a summary. See OpenAI reasoning for model and provider differences.

For more details on reasoning support across different models and providers, see the AI SDK providers documentation, including OpenAI, DeepSeek, and Anthropic.

app/api/chat/route.ts

import { streamText } from 'ai';
 
export async function POST(request: Request) {
  const { prompt } = await request.json();
 
  const result = streamText({
    model: 'openai/gpt-oss-120b',
    prompt,
    providerOptions: {
      openai: {
        reasoningEffort: 'high',
        reasoningSummary: 'detailed',
      },
    },
  });
 
  return result.toUIMessageStreamResponse();
}

For openai/gpt-6-astra, enable reasoning with a non-none effort and request a summary with reasoningSummary. Supported effort values and summary formats depend on the model. See OpenAI reasoning for model-specific options.

providerOptions: {
  openai: {
    reasoningEffort: 'high', // or 'low', 'medium', 'xhigh', 'max'
    reasoningSummary: 'detailed', // or 'auto', 'concise'
  },
}

Available providers

You can view the available models for a provider in the Model List section under the AI Gateway section in your Vercel dashboard sidebar or in the public models page.

Slug	Name	Website
`alibaba`	Alibaba Cloud	alibabacloud.com
`anthropic`	Anthropic	anthropic.com
`arcee-ai`	Arcee AI	arcee.ai
`azure`	Azure	ai.azure.com
`baseten`	Baseten	baseten.co
`bedrock`	Bedrock	aws.amazon.com
`bfl`	Black Forest Labs	bfl.ai
`boundless`	Boundless	boundless.network
`browserbase`	Browserbase	browserbase.com
`bytedance`	ByteDance	byteplus.com
`callstack`	Callstack	apex.callstack.com
`cerebras`	Cerebras	cerebras.ai
`claudeaws`	Claude Platform on AWS	aws.amazon.com
`cohere`	Cohere	cohere.com
`crusoe`	Crusoe	crusoe.ai
`deepinfra`	DeepInfra	deepinfra.com
`deepseek`	DeepSeek	deepseek.com
`digitalocean`	DigitalOcean	digitalocean.com
`exa`	Exa	exa.ai
`fireworks`	Fireworks	fireworks.ai
`fish-audio`	Fish Audio	fish.audio
`friendli`	FriendliAI	friendli.ai
`gmicloud`	GMICloud	gmicloud.ai
`google`	Google	ai.google.dev
`groq`	Groq	groq.com
`heygen`	HeyGen	heygen.com
`inception`	Inception	inceptionlabs.ai
`inceptron`	Inceptron	inceptron.io
`inference-net`	Inference.net	inference.net
`interfaze`	Interfaze	interfaze.ai
`klingai`	Kling AI	klingai.com
`liquid`	Liquid AI	liquid.ai
`meituan`	Meituan LongCat	longcat.ai
`meta`	Meta	meta.ai
`minimax`	MiniMax	minimax.io
`mistral`	Mistral	mistral.ai
`mixedbread`	Mixedbread	mixedbread.com
`modal`	Modal	modal.com
`moonshotai`	Moonshot AI	moonshot.ai
`morph`	Morph	morphllm.com
`nebius`	Nebius	nebius.com
`novita`	Novita AI	novita.ai
`openai`	OpenAI	openai.com
`parallel`	Parallel AI	parallel.ai
`parasail`	Parasail	parasail.io
`particle`	Particle.AI	particle.ai
`perplexity`	Perplexity	perplexity.ai
`poolside`	Poolside	poolside.ai
`prodia`	Prodia	prodia.com
`quiverai`	QuiverAI	quiver.ai
`recraft`	Recraft	recraft.ai
`relace`	Relace	relace.ai
`runinfra`	RunInfra	runinfra.ai
`runware`	Runware	runware.ai
`sakana`	Sakana AI	sakana.ai
`sambanova`	SambaNova	sambanova.ai
`stealth`	Stealth	vercel.com
`stepfun`	StepFun	platform.stepfun.com
`streamlake`	StreamLake	streamlake.ai
`tako`	Tako	tako.com
`tencent`	Tencent Cloud	tencentcloud.com
`thinkingmachines`	Thinking Machines	thinkingmachines.ai
`togetherai`	Together AI	together.ai
`topaz`	Topaz Labs	topazlabs.com
`typesafe-ai`	TypeSafe AI	typesafe.ai
`vertex`	Google Vertex AI	cloud.google.com
`voyage`	Voyage AI by MongoDB	voyageai.com
`wafer`	Wafer	wafer.ai
`xai`	xAI	x.ai
`xiaomi`	Xiaomi	mimo.xiaomi.com
`zai`	Z.AI	z.ai

Provider availability may vary by model. Some models may only be available through specific providers or may have different capabilities depending on the provider used.

Provider options in virtual models

A virtual model can pin provider options for all of its traffic. The virtual model's options for a provider merge with the request's, and the virtual model wins when both set the same key.

Last updated October 2, 2026

Cross-link map: AI Gateway Provider Routing and Fallbacks (/docs/ai-gateway/models-and-providers/provider-options)

From the Vercel docs graph (built 2026-10-13T05:35:52.211Z), spanning vercel.com docs + KB, nextjs.org, ai-sdk.dev, and other Vercel documentation sites. Full graph as JSON: https://vercel.com/docs/graph.json

Semantically closest pages

AI Gateway Models and Providers — Choose AI Gateway models and providers. Configure routing, fallbacks, timeouts, prompt caching, reasoning, and web searc
OpenResponses Configuration with AI Gateway — Configure provider routing, fallbacks, and restrictions using the OpenResponses API through AI Gateway.
Vercel AI Gateway: Models, Routing, and Observability — Call AI models from any infrastructure through a managed gateway. Centralize credentials, request logs, spend budgets, r
OpenAI Chat Completions Configuration with AI Gateway — Configure provider options, model fallbacks, BYOK credentials, and prompt caching through AI Gateway.
AI Gateway Disallow Prompt Training — Learn how to prevent AI providers from using your prompts and responses for model training through AI Gateway.

Prerequisites

AI Gateway — Vercel MCP tools for ai gateway.

This page links to (15)

AI SDK Providers — Learn how to use AI SDK providers.
AI Gateway — Learn how to use the AI Gateway provider with the AI SDK.
Anthropic — Learn how to use the Anthropic provider for the AI SDK.
DeepSeek — Learn how to use DeepSeek's models with the AI SDK.
OpenAI — Learn how to use the OpenAI provider for the AI SDK.
Bring Your Own Key \(BYOK\) to AI Gateway — Learn how to configure your own provider keys with AI Gateway.
AI Gateway Automatic Prompt Caching — Enable automatic prompt caching and provider-side cache affinity with AI Gateway to reduce costs and latency.
Decision Fallbacks — Escalate uncertain AI Gateway decision results to another model with confidence, probability, and compound conditions.
AI Gateway Model Fallbacks — Configure AI Gateway model fallbacks to try backup models when the primary model is unavailable. Set fallback order and
AI Gateway Provider Filtering, Ordering, and Sorting — Control AI Gateway provider routing with order, only, and sort. Set preferences, restrict providers, and rank them by co
AI Gateway Provider Timeouts — Configure AI Gateway timeouts for BYOK providers. Fail over to the next provider when a request takes too long to start
AI Gateway Anthropic Reasoning — Configure adaptive and extended thinking for Anthropic Claude models with the AI SDK and AI Gateway.
AI Gateway OpenAI Reasoning — Configure reasoning and thinking for OpenAI models with the AI SDK and AI Gateway.
AI Gateway SDKs and APIs — Connect to AI Gateway with the AI SDK, Python, REST, or compatible OpenAI, Anthropic Messages, OpenResponses, and Cohere
OpenAI Responses API with AI Gateway — Use the OpenAI Responses API with AI Gateway to generate text, call tools, stream tokens, and more across any supported

Pages that link here (24)

By site: vercel-changelog (1) · vercel-kb (1) · vercel-web (1) · vercel-docs (21)

From vercel-changelog

Routing rules now available on AI Gateway

From vercel-kb

Build AI agents with AI Gateway and AI SDK — Build AI agents on Vercel with AI Gateway and AI SDK, then make them reliable, capable, and durable with Sandbox, Chat S

From vercel-web

Zero Data Retention on AI Gateway

From vercel-docs

Hermes with AI Gateway — Connect Hermes to AI Gateway with the Vercel CLI or manual provider configuration. Set your API key, discover models, an
OpenCode with AI Gateway — Connect OpenCode to AI Gateway with one CLI command, or configure it in-app.
Getting Started with AI Gateway — Set up AI Gateway with a coding agent, route the agent through AI Gateway, or make your first request with cURL, TypeScr
Migrate to AI Gateway Using Your Coding Agent — Move your app's model calls to Vercel AI Gateway with a single coding-agent prompt, whatever provider or SDK you use tod
AI Gateway Text Generation Quickstart — Generate and stream text responses using AI Gateway.
AI Gateway Text Generation — Generate and stream text from hundreds of models through AI Gateway, with tool calling and structured output.
AI Gateway Models and Providers — Choose AI Gateway models and providers. Configure routing, fallbacks, timeouts, prompt caching, reasoning, and web searc
Decision Fallbacks — Escalate uncertain AI Gateway decision results to another model with confidence, probability, and compound conditions.
AI Gateway Model Fallbacks — Configure AI Gateway model fallbacks to try backup models when the primary model is unavailable. Set fallback order and
AI Gateway Provider Filtering, Ordering, and Sorting — Control AI Gateway provider routing with order, only, and sort. Set preferences, restrict providers, and rank them by co
AI Gateway Provider Timeouts — Configure AI Gateway timeouts for BYOK providers. Fail over to the next provider when a request takes too long to start
AI Gateway Reasoning — Discover model reasoning capabilities and configure effort across AI SDK, Chat Completions, Messages, and Responses with
AI Gateway Routing Rules — Define team-wide rules that rewrite requests from one model to another or deny specific models in AI Gateway.
AI Gateway Custom Reporting API — Query AI Gateway usage data grouped by model, user, tag, provider, or credential type using the Custom Reporting API.
AI Gateway Trace Drains — Forward an OpenTelemetry trace of every AI Gateway request to your own observability tool, and understand trace drain bi
AI SDK with AI Gateway — Build AI-powered TypeScript applications using the AI SDK with AI Gateway for unified access to 200+ models.
AI SDK for Python with AI Gateway — Build AI-powered Python applications using the AI SDK for Python with AI Gateway for unified access to 200+ models.
Anthropic Messages API with AI Gateway — Use the Anthropic Messages API with AI Gateway. Configure authentication and send requests with streaming, tools, images
Cohere Rerank API with AI Gateway — Use the Cohere-compatible Rerank API with AI Gateway to reorder documents by relevance with the Cohere SDK or plain HTTP
OpenAI Chat Completions Configuration with AI Gateway — Configure provider options, model fallbacks, BYOK credentials, and prompt caching through AI Gateway.
AI Gateway Provider Allowlist — Restrict which AI providers your team can route through AI Gateway. Available on Pro and Enterprise.

Models & Providers

Decision Fallbacks

Was this helpful?