Provider Filtering, Ordering & Sorting

By default, AI Gateway dynamically chooses providers based on recent uptime and latency. You can override this behavior to control which providers handle your requests and in what order using order, only, and sort in providerOptions.gateway. To instead filter by a capability of the model itself, see Model Filtering.

Use the order array to specify the sequence in which providers should be attempted. Providers are specified using their slug string. You can find the slugs in the table of available providers.

You can also copy the provider slug using the copy button next to a provider's name on a model's detail page:

Through the Vercel Dashboard:

1. Click the AI Gateway tab
2. Click Model List on the left
3. Click a model entry in the list

Through the AI Gateway site:

Visit a model's page on the AI Gateway models page (e.g., Claude Sonnet 5).

The bottom section of the page lists the available providers for that model. The copy button next to a provider's name will copy their slug for pasting.

1. Install the AI SDK package

   First, ensure you have the necessary package installed:

   Terminal

   pnpm install ai@latest

2. Configure the provider order in your request

   Use the providerOptions.gateway.order configuration:

   app/api/chat/route.ts

   import { streamText } from 'ai';
    
   export async function POST(request: Request) {
     const { prompt } = await request.json();
    
     const result = streamText({
       model: 'anthropic/claude-sonnet-5',
       prompt,
       providerOptions: {
         gateway: {
           order: ['bedrock', 'anthropic'], // Try Amazon Bedrock first, then Anthropic
         },
       },
     });
    
     return result.toUIMessageStreamResponse();
   }

   In this example:

   • The gateway will first attempt to use Amazon Bedrock to serve the Claude 4 Sonnet model
   • If Amazon Bedrock is unavailable or fails, it will fall back to Anthropic
   • Other providers (like Vertex AI) are still available but will only be used after the specified providers

3. Test the routing behavior

   You can monitor which provider you used by checking the provider metadata in the response.

   app/api/chat/route.ts

   import { streamText } from 'ai';
    
   export async function POST(request: Request) {
     const { prompt } = await request.json();
    
     const result = streamText({
       model: 'anthropic/claude-sonnet-5',
       prompt,
       providerOptions: {
         gateway: {
           order: ['bedrock', 'anthropic'],
         },
       },
     });
    
     // Log which provider was actually used
     console.log(JSON.stringify(await result.providerMetadata, null, 2));
    
     return result.toUIMessageStreamResponse();
   }

{
  "anthropic": {},
  "gateway": {
    "routing": {
      "originalModelId": "anthropic/claude-sonnet-5",
      "resolvedProvider": "anthropic",
      "resolvedProviderApiModelId": "claude-sonnet-5",
      "fallbacksAvailable": ["bedrock", "vertex"],
      "planningReasoning": "System credentials planned for: anthropic. Total execution order: anthropic(system)",
      "canonicalSlug": "anthropic/claude-sonnet-5",
      "finalProvider": "anthropic",
      "modelAttemptCount": 1,
      "modelAttempts": [
        {
          "modelId": "anthropic:claude-sonnet-5",
          "canonicalSlug": "anthropic/claude-sonnet-5",
          "success": true,
          "providerAttemptCount": 1,
          "providerAttempts": [
            {
              "provider": "anthropic",
              "providerApiModelId": "claude-sonnet-5",
              "credentialType": "system",
              "success": true,
              "startTime": 458753.407267,
              "endTime": 459891.705775
            }
          ]
        }
      ],
      "totalProviderAttemptCount": 1
    },
    "cost": "0.0045405",
    "marketCost": "0.0045405",
    "generationId": "gen_01A2B3C4D5E6F7G8H9J0K1L2M"
  }
}

The gateway.cost value is the inference cost for this request, returned as a decimal string. It does not include other charges that may apply (for example, Custom Reporting writes or Zero Data Retention surcharges). The gateway.marketCost represents the market rate cost for the inference. The gateway.generationId is a unique identifier for this generation that can be used with the Generation Lookup API. For more on pricing see Pricing.

In cases where your request encounters issues with one or more providers or if your BYOK credentials fail, you'll find error detail in the providerAttempts array within each entry of modelAttempts:

"modelAttempts": [
  {
    "modelId": "novita:zai-org/glm-5",
    "canonicalSlug": "zai/glm-5",
    "success": true,
    "providerAttemptCount": 2,
    "providerAttempts": [
      {
        "provider": "novita",
        "providerApiModelId": "zai-org/glm-5",
        "credentialType": "byok",
        "success": false,
        "error": "Unauthorized",
        "startTime": 1754639042520,
        "endTime": 1754639042710
      },
      {
        "provider": "novita",
        "providerApiModelId": "zai-org/glm-5",
        "credentialType": "system",
        "success": true,
        "startTime": 1754639042710,
        "endTime": 1754639043353
      }
    ]
  }
]

Use the only array to restrict routing to a specific subset of providers. Providers are specified by their slug and are matched against the model's available providers.

import { streamText } from 'ai';
 
export async function POST(request: Request) {
  const { prompt } = await request.json();
 
  const result = streamText({
    model: 'anthropic/claude-sonnet-5',
    prompt,
    providerOptions: {
      gateway: {
        only: ['bedrock', 'anthropic'], // Only consider these providers.
        // This model is also available via 'vertex', but it won't be considered.
      },
    },
  });
 
  return result.toUIMessageStreamResponse();
}

In this example:

• Restriction: Only bedrock and anthropic will be considered for routing and fallbacks.
• Error on mismatch: If none of the specified providers are available for the model, the request fails with an error indicating the allowed providers.

When both only and order are provided, the only filter is applied first to define the allowed set, and then order defines the priority within that filtered set. Practically, the end result is the same as taking your order list and intersecting it with the only list.

import { streamText } from 'ai';
 
export async function POST(request: Request) {
  const { prompt } = await request.json();
 
  const result = streamText({
    model: 'anthropic/claude-sonnet-5',
    prompt,
    providerOptions: {
      gateway: {
        only: ['anthropic', 'vertex'],
        order: ['vertex', 'bedrock', 'anthropic'],
      },
    },
  });
 
  return result.toUIMessageStreamResponse();
}

The final order will be vertex → anthropic (providers listed in order but not in only are ignored).

Use the sort option to rank providers by a performance or cost metric. The gateway sorts the available providers by the chosen metric and tries them in that order, falling back through the list if a provider fails.

import { streamText } from 'ai';
 
export async function POST(request: Request) {
  const { prompt } = await request.json();
 
  const result = streamText({
    model: 'anthropic/claude-sonnet-5',
    prompt,
    providerOptions: {
      gateway: {
        sort: 'cost', // Use the lowest cost provider first
      },
    },
  });
 
  return result.toUIMessageStreamResponse();
}

import { streamText } from 'ai';
 
export async function POST(request: Request) {
  const { prompt } = await request.json();
 
  const result = streamText({
    model: 'anthropic/claude-sonnet-5',
    prompt,
    providerOptions: {
      gateway: {
        sort: 'ttft', // Use the fastest provider first
      },
    },
  });
 
  return result.toUIMessageStreamResponse();
}

You can combine sort with order and only. When combined with order, the providers you specify in order are promoted to the front of the list, while the remaining providers follow the sorted order. When combined with only, sorting is applied within the restricted set of providers.

import { streamText } from 'ai';
 
export async function POST(request: Request) {
  const { prompt } = await request.json();
 
  const result = streamText({
    model: 'anthropic/claude-sonnet-5',
    prompt,
    providerOptions: {
      gateway: {
        only: ['anthropic', 'bedrock', 'vertex'],
        sort: 'tps', // Among these three, try the fastest throughput first
      },
    },
  });
 
  return result.toUIMessageStreamResponse();
}

When sort is active, the response's provider metadata includes a sort object inside gateway.routing:

{
  "gateway": {
    "routing": {
      "sort": {
        "option": "cost",
        "executionOrder": ["anthropic", "bedrock", "vertex"],
        "metrics": {
          "anthropic": 0.003,
          "bedrock": 0.003,
          "vertex": 0.005
        },
        "deprioritizedProviders": []
      }
    }
  }
}

The gateway uses provider health status as a guard rail when sorting:

• Healthy providers are sorted purely by the chosen metric.
• Degraded or recovering providers receive a penalty to their metric score, pushing them lower in the sort order.
• Down providers are always sorted last, regardless of their metric values.

This means sort optimizes for your chosen metric while still avoiding unhealthy providers.

All options are set under providerOptions.gateway in the AI SDK, or under providerOptions in the REST API / OpenAI-compatible Chat Completions API. The Chat Completions API also accepts a top-level provider shorthand (e.g., "provider": { "sort": "tps" }). See Available Providers for the full list of provider slugs.

To filter by a capability of the model itself (rather than by provider), see Model Filtering.