Filtering, Ordering & Sorting

AI Gateway Provider Filtering, Ordering, and Sorting

By default, AI Gateway dynamically chooses providers based on recent uptime and latency. You can override this behavior to control which providers handle your requests and in what order using order, only, and sort in providerOptions.gateway. To instead filter by a capability of the model itself, see Model Filtering.

Provider ordering

Use the order array to specify the sequence in which providers should be attempted. Provider ordering works with a model served by more than one provider, such as anthropic/claude-sonnet-5.

Providers are specified using their slug string. You can find the slugs in the table of available providers. To find which providers serve a model programmatically, get its provider endpoints.

You can also copy the provider slug using the copy button next to a provider's name on a model's detail page:

Through the Vercel Dashboard:

Click the AI Gateway tab
Click Model List on the left
Click a model entry in the list

Through the AI Gateway site:

Visit a model's page on the AI Gateway models page (e.g., Claude Sonnet 5).

The bottom section of the page lists the available providers for that model. The copy button next to a provider's name will copy their slug for pasting.

Getting started

Install the AI SDK package
First, ensure you have the necessary package installed:
Terminal
```
pnpm install ai@latest
```

Configure the provider order in your request

Use the providerOptions.gateway.order configuration:

These examples use AI SDK 7 and the AI SDK for Python beta. Set AI_GATEWAY_API_KEY before running them. See API format differences for setup, request fields, and response handling.

See the AI SDK routing-options reference for SDK configuration and usage.

provider-ordering.ts

import { streamText } from 'ai';
 
const result = streamText({
  model: 'anthropic/claude-sonnet-5',
  prompt: 'Explain quantum computing in two sentences.',
  providerOptions: {
    gateway: {
      order: ['bedrock', 'anthropic'],
    },
  },
});
 
for await (const chunk of result.textStream) {
  process.stdout.write(chunk);
}

provider-ordering_ai.py

import asyncio
import ai
 
async def main():
    model = ai.get_model("anthropic/claude-sonnet-5")
    messages = [ai.user_message("Explain quantum computing in two sentences.")]
    params = ai.InferenceRequestParams(
        extra_body={"providerOptions": {"gateway": {"order": ["bedrock", "anthropic"]}}}
    )
    async with ai.stream(model, messages, params=params) as stream:
        async for event in stream:
            if isinstance(event, ai.events.TextDelta):
                print(event.chunk, end="", flush=True)
    print()
 
asyncio.run(main())

provider-ordering-chat.ts

import OpenAI from 'openai';
 
const client = new OpenAI({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh/v1',
});
 
const response = await client.chat.completions.create({
  model: 'anthropic/claude-sonnet-5',
  messages: [
    {
      role: 'user',
      content: 'Explain quantum computing in two sentences.',
    },
  ],
  // AI Gateway extension fields are not included in the upstream SDK types.
  ...{
    providerOptions: {
      gateway: {
        order: ['bedrock', 'anthropic'],
      },
    },
  },
  stream: true,
});
 
for await (const event of response) {
  process.stdout.write(event.choices[0]?.delta.content ?? '');
}

provider-ordering_chat.py

import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.environ["AI_GATEWAY_API_KEY"],
    base_url="https://ai-gateway.vercel.sh/v1",
)
 
response = client.chat.completions.create(
    model="anthropic/claude-sonnet-5",
    messages=[{"role": "user", "content": "Explain quantum computing in two sentences."}],
    extra_body={"providerOptions": {"gateway": {"order": ["bedrock", "anthropic"]}}},
    stream=True,
)
 
for event in response:
    if event.choices:
        print(event.choices[0].delta.content or "", end="", flush=True)

provider-ordering-chat.sh

curl --fail-with-body --no-buffer https://ai-gateway.vercel.sh/v1/chat/completions \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "anthropic/claude-sonnet-5",
  "messages": [
    {
      "role": "user",
      "content": "Explain quantum computing in two sentences."
    }
  ],
  "providerOptions": {
    "gateway": {
      "order": [
        "bedrock",
        "anthropic"
      ]
    }
  },
  "stream": true
}'

provider-ordering-messages.ts

import Anthropic from '@anthropic-ai/sdk';
 
const client = new Anthropic({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh',
});
 
const response = await client.messages.create({
  model: 'anthropic/claude-sonnet-5',
  messages: [
    {
      role: 'user',
      content: 'Explain quantum computing in two sentences.',
    },
  ],
  max_tokens: 1024,
  ...{
    providerOptions: {
      gateway: {
        order: ['bedrock', 'anthropic'],
      },
    },
  },
  stream: true,
});
 
for await (const event of response) {
  if (
    event.type === 'content_block_delta' &&
    event.delta.type === 'text_delta'
  ) {
    process.stdout.write(event.delta.text);
  }
}

provider-ordering_messages.py

import os
from anthropic import Anthropic
 
client = Anthropic(
    api_key=os.environ["AI_GATEWAY_API_KEY"],
    base_url="https://ai-gateway.vercel.sh",
)
 
response = client.messages.create(
    model="anthropic/claude-sonnet-5",
    messages=[{"role": "user", "content": "Explain quantum computing in two sentences."}],
    max_tokens=1024,
    extra_body={"providerOptions": {"gateway": {"order": ["bedrock", "anthropic"]}}},
    stream=True,
)
 
for event in response:
    if event.type == "content_block_delta" and event.delta.type == "text_delta":
        print(event.delta.text, end="", flush=True)

provider-ordering-messages.sh

curl --fail-with-body --no-buffer https://ai-gateway.vercel.sh/v1/messages \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
  "model": "anthropic/claude-sonnet-5",
  "messages": [
    {
      "role": "user",
      "content": "Explain quantum computing in two sentences."
    }
  ],
  "max_tokens": 1024,
  "providerOptions": {
    "gateway": {
      "order": [
        "bedrock",
        "anthropic"
      ]
    }
  },
  "stream": true
}'

provider-ordering-responses.ts

import OpenAI from 'openai';
 
const client = new OpenAI({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh/v1',
});
 
const response = await client.responses.create({
  model: 'anthropic/claude-sonnet-5',
  input: 'Explain quantum computing in two sentences.',
  ...{
    providerOptions: {
      gateway: {
        order: ['bedrock', 'anthropic'],
      },
    },
  },
  stream: true,
});
 
for await (const event of response) {
  if (event.type === 'response.output_text.delta') {
    process.stdout.write(event.delta);
  }
}

provider-ordering_responses.py

import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.environ["AI_GATEWAY_API_KEY"],
    base_url="https://ai-gateway.vercel.sh/v1",
)
 
response = client.responses.create(
    model="anthropic/claude-sonnet-5",
    input="Explain quantum computing in two sentences.",
    extra_body={"providerOptions": {"gateway": {"order": ["bedrock", "anthropic"]}}},
    stream=True,
)
 
for event in response:
    if event.type == "response.output_text.delta":
        print(event.delta, end="", flush=True)

provider-ordering-responses.sh

curl --fail-with-body --no-buffer https://ai-gateway.vercel.sh/v1/responses \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "anthropic/claude-sonnet-5",
  "input": "Explain quantum computing in two sentences.",
  "providerOptions": {
    "gateway": {
      "order": [
        "bedrock",
        "anthropic"
      ]
    }
  },
  "stream": true
}'

In this example:

AI Gateway will first attempt to use Amazon Bedrock to serve the Claude Sonnet 5 model
If Amazon Bedrock is unavailable or fails, it will fall back to Anthropic
Other providers (like Vertex AI) are still available but will only be used after the specified providers

Test the routing behavior

Read providerMetadata.gateway.routing.finalProvider to print the provider that served the response:

provider-ordering-result.ts

import { generateText } from 'ai';
 
const result = await generateText({
  model: 'anthropic/claude-sonnet-5',
  prompt: 'Explain quantum computing in two sentences.',
  providerOptions: {
    gateway: {
      order: ['bedrock', 'anthropic'],
    },
  },
});
 
const gatewayMetadata = result.providerMetadata?.gateway as
  | { routing?: { finalProvider?: string } }
  | undefined;
 
console.log(result.text);
console.log('Provider:', gatewayMetadata?.routing?.finalProvider ?? 'unknown');

Provider metadata output

{
  "anthropic": {},
  "gateway": {
    "routing": {
      "originalModelId": "anthropic/claude-sonnet-5",
      "resolvedProvider": "anthropic",
      "resolvedProviderApiModelId": "claude-sonnet-5",
      "fallbacksAvailable": ["bedrock", "vertex"],
      "planningReasoning": "System credentials planned for: anthropic. Total execution order: anthropic(system)",
      "canonicalSlug": "anthropic/claude-sonnet-5",
      "finalProvider": "anthropic",
      "modelAttemptCount": 1,
      "modelAttempts": [
        {
          "modelId": "anthropic:claude-sonnet-5",
          "canonicalSlug": "anthropic/claude-sonnet-5",
          "success": true,
          "providerAttemptCount": 1,
          "providerAttempts": [
            {
              "provider": "anthropic",
              "providerApiModelId": "claude-sonnet-5",
              "credentialType": "system",
              "success": true,
              "startTime": 458753.407267,
              "endTime": 459891.705775
            }
          ]
        }
      ],
      "totalProviderAttemptCount": 1
    },
    "cost": "0.0045405",
    "marketCost": "0.0045405",
    "generationId": "gen_01A2B3C4D5E6F7G8H9J0K1L2M"
  }
}

The gateway.cost value is the inference cost for this request, returned as a decimal string. It does not include other charges that may apply (for example, Custom Reporting writes or Zero Data Retention surcharges). The gateway.marketCost represents the market rate cost for the inference. The gateway.generationId is a unique identifier for this generation that can be used with the Generation Lookup API. For more on pricing see Pricing.

In cases where your request encounters issues with one or more providers or if your BYOK credentials fail, you'll find error detail in the providerAttempts array within each entry of modelAttempts:

"modelAttempts": [
  {
    "modelId": "novita:zai-org/glm-5",
    "canonicalSlug": "zai/glm-5",
    "success": true,
    "providerAttemptCount": 2,
    "providerAttempts": [
      {
        "provider": "novita",
        "providerApiModelId": "zai-org/glm-5",
        "credentialType": "byok",
        "success": false,
        "error": "Unauthorized",
        "startTime": 1754639042520,
        "endTime": 1754639042710
      },
      {
        "provider": "novita",
        "providerApiModelId": "zai-org/glm-5",
        "credentialType": "system",
        "success": true,
        "startTime": 1754639042710,
        "endTime": 1754639043353
      }
    ]
  }
]

Provider filtering

Restrict providers with the `only` filter

Use the only array to restrict routing to a specific subset of providers. Providers are specified by their slug and are matched against the model's available providers.

These examples use AI SDK 7 and the AI SDK for Python beta. Set AI_GATEWAY_API_KEY before running them. See API format differences for setup, request fields, and response handling.

provider-filtering.ts

import { generateText } from 'ai';
 
const { text } = await generateText({
  model: 'anthropic/claude-sonnet-5',
  prompt: 'Explain quantum computing in two sentences.',
  providerOptions: {
    gateway: {
      only: ['bedrock', 'anthropic'],
    },
  },
});
 
console.log(text);

provider-filtering_ai.py

import asyncio
import ai
 
async def main():
    model = ai.get_model("anthropic/claude-sonnet-5")
    messages = [ai.user_message("Explain quantum computing in two sentences.")]
    params = ai.InferenceRequestParams(
        extra_body={"providerOptions": {"gateway": {"only": ["bedrock", "anthropic"]}}}
    )
    async with ai.stream(model, messages, params=params) as stream:
        async for event in stream:
            if isinstance(event, ai.events.TextDelta):
                print(event.chunk, end="", flush=True)
    print()
 
asyncio.run(main())

provider-filtering-chat.ts

import OpenAI from 'openai';
 
const client = new OpenAI({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh/v1',
});
 
const response = await client.chat.completions.create({
  model: 'anthropic/claude-sonnet-5',
  messages: [
    {
      role: 'user',
      content: 'Explain quantum computing in two sentences.',
    },
  ],
  ...{
    providerOptions: {
      gateway: {
        only: ['bedrock', 'anthropic'],
      },
    },
  },
});
 
console.log(response.choices[0]?.message.content);

provider-filtering_chat.py

import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.environ["AI_GATEWAY_API_KEY"],
    base_url="https://ai-gateway.vercel.sh/v1",
)
 
response = client.chat.completions.create(
    model="anthropic/claude-sonnet-5",
    messages=[{"role": "user", "content": "Explain quantum computing in two sentences."}],
    extra_body={"providerOptions": {"gateway": {"only": ["bedrock", "anthropic"]}}},
)
 
print(response.choices[0].message.content)

provider-filtering-chat.sh

curl --fail-with-body https://ai-gateway.vercel.sh/v1/chat/completions \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "anthropic/claude-sonnet-5",
  "messages": [
    {
      "role": "user",
      "content": "Explain quantum computing in two sentences."
    }
  ],
  "providerOptions": {
    "gateway": {
      "only": [
        "bedrock",
        "anthropic"
      ]
    }
  }
}'

provider-filtering-messages.ts

import Anthropic from '@anthropic-ai/sdk';
 
const client = new Anthropic({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh',
});
 
const response = await client.messages.create({
  model: 'anthropic/claude-sonnet-5',
  messages: [
    {
      role: 'user',
      content: 'Explain quantum computing in two sentences.',
    },
  ],
  max_tokens: 1024,
  ...{
    providerOptions: {
      gateway: {
        only: ['bedrock', 'anthropic'],
      },
    },
  },
});
 
for (const block of response.content) {
  if (block.type === 'text') console.log(block.text);
}

provider-filtering_messages.py

import os
from anthropic import Anthropic
 
client = Anthropic(
    api_key=os.environ["AI_GATEWAY_API_KEY"],
    base_url="https://ai-gateway.vercel.sh",
)
 
response = client.messages.create(
    model="anthropic/claude-sonnet-5",
    messages=[{"role": "user", "content": "Explain quantum computing in two sentences."}],
    max_tokens=1024,
    extra_body={"providerOptions": {"gateway": {"only": ["bedrock", "anthropic"]}}},
)
 
for block in response.content:
    if block.type == "text":
        print(block.text)

provider-filtering-messages.sh

curl --fail-with-body https://ai-gateway.vercel.sh/v1/messages \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
  "model": "anthropic/claude-sonnet-5",
  "messages": [
    {
      "role": "user",
      "content": "Explain quantum computing in two sentences."
    }
  ],
  "max_tokens": 1024,
  "providerOptions": {
    "gateway": {
      "only": [
        "bedrock",
        "anthropic"
      ]
    }
  }
}'

provider-filtering-responses.ts

import OpenAI from 'openai';
 
const client = new OpenAI({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh/v1',
});
 
const response = await client.responses.create({
  model: 'anthropic/claude-sonnet-5',
  input: 'Explain quantum computing in two sentences.',
  ...{
    providerOptions: {
      gateway: {
        only: ['bedrock', 'anthropic'],
      },
    },
  },
});
 
console.log(response.output_text);

provider-filtering_responses.py

import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.environ["AI_GATEWAY_API_KEY"],
    base_url="https://ai-gateway.vercel.sh/v1",
)
 
response = client.responses.create(
    model="anthropic/claude-sonnet-5",
    input="Explain quantum computing in two sentences.",
    extra_body={"providerOptions": {"gateway": {"only": ["bedrock", "anthropic"]}}},
)
 
print(response.output_text)

provider-filtering-responses.sh

curl --fail-with-body https://ai-gateway.vercel.sh/v1/responses \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "anthropic/claude-sonnet-5",
  "input": "Explain quantum computing in two sentences.",
  "providerOptions": {
    "gateway": {
      "only": [
        "bedrock",
        "anthropic"
      ]
    }
  }
}'

In this example:

Restriction: Only bedrock and anthropic will be considered for routing and fallbacks.
Error on mismatch: If none of the specified providers are available for the model, the request fails with an error indicating the allowed providers.

Using `only` together with `order`

When both only and order are provided, the only filter is applied first to define the allowed set, and then order defines the priority within that filtered set. Practically, the end result is the same as taking your order list and intersecting it with the only list.

app/api/chat/route.ts

import { streamText } from 'ai';
 
export async function POST(request: Request) {
  const { prompt } = await request.json();
 
  const result = streamText({
    model: 'anthropic/claude-sonnet-5',
    prompt,
    providerOptions: {
      gateway: {
        only: ['anthropic', 'vertex'],
        order: ['vertex', 'bedrock', 'anthropic'],
      },
    },
  });
 
  return result.toUIMessageStreamResponse();
}

The final order will be vertex → anthropic (providers listed in order but not in only are ignored).

Provider sorting

Use the sort option to rank providers by a performance or cost metric. The gateway sorts the available providers by the chosen metric and tries them in that order, falling back through the list if a provider fails.

Value	Description	Direction
`'cost'`	Sort by estimated cost	Lowest cost first
`'ttft'`	Sort by time to first token (median, in ms)	Lowest latency first
`'tps'`	Sort by tokens per second throughput (median)	Highest first

Sort by cost

These examples use AI SDK 7 and the AI SDK for Python beta. Set AI_GATEWAY_API_KEY before running them. See API format differences for setup, request fields, and response handling.

provider-sorting.ts

import { generateText } from 'ai';
 
const { text } = await generateText({
  model: 'anthropic/claude-sonnet-5',
  prompt: 'Explain quantum computing in two sentences.',
  providerOptions: {
    gateway: {
      sort: 'cost',
    },
  },
});
 
console.log(text);

provider-sorting_ai.py

import asyncio
import ai
 
async def main():
    model = ai.get_model("anthropic/claude-sonnet-5")
    messages = [ai.user_message("Explain quantum computing in two sentences.")]
    params = ai.InferenceRequestParams(
        extra_body={"providerOptions": {"gateway": {"sort": "cost"}}}
    )
    async with ai.stream(model, messages, params=params) as stream:
        async for event in stream:
            if isinstance(event, ai.events.TextDelta):
                print(event.chunk, end="", flush=True)
    print()
 
asyncio.run(main())

provider-sorting-chat.ts

import OpenAI from 'openai';
 
const client = new OpenAI({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh/v1',
});
 
const response = await client.chat.completions.create({
  model: 'anthropic/claude-sonnet-5',
  messages: [
    {
      role: 'user',
      content: 'Explain quantum computing in two sentences.',
    },
  ],
  ...{
    providerOptions: {
      gateway: {
        sort: 'cost',
      },
    },
  },
});
 
console.log(response.choices[0]?.message.content);

provider-sorting_chat.py

import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.environ["AI_GATEWAY_API_KEY"],
    base_url="https://ai-gateway.vercel.sh/v1",
)
 
response = client.chat.completions.create(
    model="anthropic/claude-sonnet-5",
    messages=[{"role": "user", "content": "Explain quantum computing in two sentences."}],
    extra_body={"providerOptions": {"gateway": {"sort": "cost"}}},
)
 
print(response.choices[0].message.content)

provider-sorting-chat.sh

curl --fail-with-body https://ai-gateway.vercel.sh/v1/chat/completions \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "anthropic/claude-sonnet-5",
  "messages": [
    {
      "role": "user",
      "content": "Explain quantum computing in two sentences."
    }
  ],
  "providerOptions": {
    "gateway": {
      "sort": "cost"
    }
  }
}'

provider-sorting-messages.ts

import Anthropic from '@anthropic-ai/sdk';
 
const client = new Anthropic({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh',
});
 
const response = await client.messages.create({
  model: 'anthropic/claude-sonnet-5',
  messages: [
    {
      role: 'user',
      content: 'Explain quantum computing in two sentences.',
    },
  ],
  max_tokens: 1024,
  ...{
    providerOptions: {
      gateway: {
        sort: 'cost',
      },
    },
  },
});
 
for (const block of response.content) {
  if (block.type === 'text') console.log(block.text);
}

provider-sorting_messages.py

import os
from anthropic import Anthropic
 
client = Anthropic(
    api_key=os.environ["AI_GATEWAY_API_KEY"],
    base_url="https://ai-gateway.vercel.sh",
)
 
response = client.messages.create(
    model="anthropic/claude-sonnet-5",
    messages=[{"role": "user", "content": "Explain quantum computing in two sentences."}],
    max_tokens=1024,
    extra_body={"providerOptions": {"gateway": {"sort": "cost"}}},
)
 
for block in response.content:
    if block.type == "text":
        print(block.text)

provider-sorting-messages.sh

curl --fail-with-body https://ai-gateway.vercel.sh/v1/messages \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
  "model": "anthropic/claude-sonnet-5",
  "messages": [
    {
      "role": "user",
      "content": "Explain quantum computing in two sentences."
    }
  ],
  "max_tokens": 1024,
  "providerOptions": {
    "gateway": {
      "sort": "cost"
    }
  }
}'

provider-sorting-responses.ts

import OpenAI from 'openai';
 
const client = new OpenAI({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh/v1',
});
 
const response = await client.responses.create({
  model: 'anthropic/claude-sonnet-5',
  input: 'Explain quantum computing in two sentences.',
  ...{
    providerOptions: {
      gateway: {
        sort: 'cost',
      },
    },
  },
});
 
console.log(response.output_text);

provider-sorting_responses.py

import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.environ["AI_GATEWAY_API_KEY"],
    base_url="https://ai-gateway.vercel.sh/v1",
)
 
response = client.responses.create(
    model="anthropic/claude-sonnet-5",
    input="Explain quantum computing in two sentences.",
    extra_body={"providerOptions": {"gateway": {"sort": "cost"}}},
)
 
print(response.output_text)

provider-sorting-responses.sh

curl --fail-with-body https://ai-gateway.vercel.sh/v1/responses \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "anthropic/claude-sonnet-5",
  "input": "Explain quantum computing in two sentences.",
  "providerOptions": {
    "gateway": {
      "sort": "cost"
    }
  }
}'

Sort by latency

app/api/chat/route.ts

import { streamText } from 'ai';
 
export async function POST(request: Request) {
  const { prompt } = await request.json();
 
  const result = streamText({
    model: 'anthropic/claude-sonnet-5',
    prompt,
    providerOptions: {
      gateway: {
        sort: 'ttft', // Use the fastest provider first
      },
    },
  });
 
  return result.toUIMessageStreamResponse();
}

Combining `sort` with `order` and `only`

You can combine sort with order and only. When combined with order, the providers you specify in order are promoted to the front of the list, while the remaining providers follow the sorted order. When combined with only, sorting is applied within the restricted set of providers.

app/api/chat/route.ts

import { streamText } from 'ai';
 
export async function POST(request: Request) {
  const { prompt } = await request.json();
 
  const result = streamText({
    model: 'anthropic/claude-sonnet-5',
    prompt,
    providerOptions: {
      gateway: {
        only: ['anthropic', 'bedrock', 'vertex'],
        sort: 'tps', // Among these three, try the fastest throughput first
      },
    },
  });
 
  return result.toUIMessageStreamResponse();
}

Sort metadata

When sort is active, the response's provider metadata includes a sort object inside gateway.routing:

{
  "gateway": {
    "routing": {
      "sort": {
        "option": "cost",
        "executionOrder": ["anthropic", "bedrock", "vertex"],
        "metrics": {
          "anthropic": 0.003,
          "bedrock": 0.003,
          "vertex": 0.005
        },
        "deprioritizedProviders": []
      }
    }
  }
}

Field	Description
`option`	The sort metric used (`cost`, `ttft`, or `tps`)
`executionOrder`	Providers in the order they were attempted after sorting
`metrics`	Per-provider metric values used for ranking (`null` if no data available)
`deprioritizedProviders`	Providers that were penalized due to degraded health

How sort interacts with provider health

The gateway uses provider health status as a guard rail when sorting:

Healthy providers are sorted purely by the chosen metric.
Degraded or recovering providers receive a penalty to their metric score, pushing them lower in the sort order.
Down providers are always sorted last, regardless of their metric values.

This means sort optimizes for your chosen metric while still avoiding unhealthy providers.

Quick reference

Option	Type	Description
`order`	`string[]`	Provider slugs in the order they should be attempted
`only`	`string[]`	Restrict routing to only these provider slugs
`sort`	`'cost'` \| `'ttft'` \| `'tps'`	Sort providers by cost, time to first token, or tokens per second

All options are set under providerOptions.gateway in the AI SDK, or under providerOptions in the REST API / OpenAI-compatible Chat Completions API. The Chat Completions API also accepts a top-level provider shorthand (e.g., "provider": { "sort": "tps" }). See Available Providers for the full list of provider slugs.

To filter by a capability of the model itself (rather than by provider), see Model Filtering.

Provider selection in virtual models

A virtual model can pin provider order and provider-only lists for all of its traffic. The virtual model's values override the request's order and only.

Last updated September 11, 2026

Cross-link map: AI Gateway Provider Filtering, Ordering, and Sorting (/docs/ai-gateway/models-and-providers/provider-filtering-and-ordering)

From the Vercel docs graph (built 2026-10-06T09:42:32.185Z), spanning vercel.com docs + KB, nextjs.org, ai-sdk.dev, and other Vercel documentation sites. Full graph as JSON: https://vercel.com/docs/graph.json

Semantically closest pages

Call AI Gateway Chat Completions with REST — Use AI Gateway API directly without client libraries using curl and fetch.
AI Gateway Service Tiers — Control processing priority and cost for OpenAI, Google AI Studio, Google Vertex AI, and SpaceXAI models using service t
OpenAI Chat Completions Configuration with AI Gateway — Configure provider options, model fallbacks, BYOK credentials, and prompt caching through AI Gateway.
AI Gateway Provider Routing and Fallbacks — Configure provider routing, ordering, and fallback behavior in Vercel AI Gateway.
OpenResponses Configuration with AI Gateway — Configure provider routing, fallbacks, and restrictions using the OpenResponses API through AI Gateway.

Prerequisites

AI Gateway — Vercel MCP tools for ai gateway.

This page links to (6)

AI Gateway — Learn how to use the AI Gateway provider with the AI SDK.
AI Gateway Model Filtering — Restrict AI Gateway routing to models that have specific capabilities using the has option.
AI Gateway Provider Routing and Fallbacks — Configure provider routing, ordering, and fallback behavior in Vercel AI Gateway.
AI Gateway Pricing — Understand AI Gateway token pricing, free and paid credits, BYOK costs, add-on charges, and payment fees. Manage credit
AI Gateway SDKs and APIs — Connect to AI Gateway with the AI SDK, Python, REST, or compatible OpenAI, Anthropic Messages, OpenResponses, and Cohere
AI Gateway REST API Reference — Reference for AI Gateway REST endpoints: models, usage, generations, and reporting.

Pages that link here (12)

By site: vercel-changelog (2) · vercel-docs (10)

From vercel-changelog

From vercel-docs

Vercel AI Gateway: Models, Routing, and Observability — Call AI models from any infrastructure through a managed gateway. Centralize credentials, request logs, spend budgets, r
TanStack AI with AI Gateway — Connect TanStack AI to Vercel AI Gateway with the official adapter to authenticate requests and stream responses from AI
AI Gateway Models and Providers — Choose AI Gateway models and providers. Configure routing, fallbacks, timeouts, prompt caching, reasoning, and web searc
AI Gateway Automatic Prompt Caching — Enable automatic prompt caching and provider-side cache affinity with AI Gateway to reduce costs and latency.
AI Gateway Model Filtering — Restrict AI Gateway routing to models that have specific capabilities using the has option.
AI Gateway Provider Routing and Fallbacks — Configure provider routing, ordering, and fallback behavior in Vercel AI Gateway.
AI Gateway Uptime and Provider Status — See how uptime and status track AI Gateway provider health and end-to-end request success.
AI Gateway SDKs and APIs — Connect to AI Gateway with the AI SDK, Python, REST, or compatible OpenAI, Anthropic Messages, OpenResponses, and Cohere
AI SDK for Python with AI Gateway — Build AI-powered Python applications using the AI SDK for Python with AI Gateway for unified access to 200+ models.
OpenAI Chat Completions Configuration with AI Gateway — Configure provider options, model fallbacks, BYOK credentials, and prompt caching through AI Gateway.

Automatic Caching

Model Filtering

Was this helpful?