Skip to content
Docs

OpenResponses Configuration with AI Gateway

The OpenResponses API lets you configure AI Gateway behavior using providerOptions. The gateway namespace gives you control over provider routing, fallbacks, and restrictions.

Model fallbacks

Set up automatic fallbacks so if your primary model is unavailable, requests route to backup models in order. Use the models array to specify the fallback chain.

fallbacks.ts
const apiKey = process.env.AI_GATEWAY_API_KEY;
 
const response = await fetch('https://ai-gateway.vercel.sh/v1/responses', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    Authorization: `Bearer ${apiKey}`,
  },
  body: JSON.stringify({
    model: 'anthropic/claude-opus-5',
    input: [
      {
        type: 'message',
        role: 'user',
        content: 'Tell me a fun fact about octopuses.',
      },
    ],
    providerOptions: {
      gateway: {
        models: ['openai/gpt-6-astra', 'google/gemini-3.1-pro-preview'],
      },
    },
  }),
});
 
if (!response.ok) throw new Error(await response.text());
console.log(await response.json());
request.py
import json
import os
import urllib.request
 
request = urllib.request.Request(
    'https://ai-gateway.vercel.sh/v1/responses',
    data=json.dumps({'model': 'anthropic/claude-opus-5', 'input': [{'type': 'message', 'role': 'user', 'content': 'Tell me a fun fact about octopuses.'}], 'providerOptions': {'gateway': {'models': ['openai/gpt-6-astra', 'google/gemini-3.1-pro-preview']}}}).encode(),
    headers={'Content-Type': 'application/json', 'Authorization': "Bearer " + os.environ["AI_GATEWAY_API_KEY"]},
)
with urllib.request.urlopen(request) as response:
    print(json.load(response))
request.sh
curl --fail-with-body https://ai-gateway.vercel.sh/v1/responses \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "anthropic/claude-opus-5",
  "input": [
    {
      "type": "message",
      "role": "user",
      "content": "Tell me a fun fact about octopuses."
    }
  ],
  "providerOptions": {
    "gateway": {
      "models": [
        "openai/gpt-6-astra",
        "google/gemini-3.1-pro-preview"
      ]
    }
  }
}'

Provider routing

Control the order in which providers are tried using the order array. AI Gateway will attempt providers in the specified order until one succeeds.

routing.ts
const apiKey = process.env.AI_GATEWAY_API_KEY;
 
const response = await fetch('https://ai-gateway.vercel.sh/v1/responses', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    Authorization: `Bearer ${apiKey}`,
  },
  body: JSON.stringify({
    model: 'google/gemini-3.1-pro-preview',
    input: [
      {
        type: 'message',
        role: 'user',
        content: 'Explain quantum computing in one sentence.',
      },
    ],
    providerOptions: {
      gateway: {
        order: ['google', 'vertex'],
      },
    },
  }),
});
 
if (!response.ok) throw new Error(await response.text());
console.log(await response.json());
request.py
import json
import os
import urllib.request
 
request = urllib.request.Request(
    'https://ai-gateway.vercel.sh/v1/responses',
    data=json.dumps({'model': 'google/gemini-3.1-pro-preview', 'input': [{'type': 'message', 'role': 'user', 'content': 'Explain quantum computing in one sentence.'}], 'providerOptions': {'gateway': {'order': ['google', 'vertex']}}}).encode(),
    headers={'Content-Type': 'application/json', 'Authorization': "Bearer " + os.environ["AI_GATEWAY_API_KEY"]},
)
with urllib.request.urlopen(request) as response:
    print(json.load(response))
request.sh
curl --fail-with-body https://ai-gateway.vercel.sh/v1/responses \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "google/gemini-3.1-pro-preview",
  "input": [
    {
      "type": "message",
      "role": "user",
      "content": "Explain quantum computing in one sentence."
    }
  ],
  "providerOptions": {
    "gateway": {
      "order": [
        "google",
        "vertex"
      ]
    }
  }
}'

Provider restriction

Restrict requests to specific providers using the only array. This ensures your requests only go to approved providers, which can be useful for compliance or cost control.

restriction.ts
const apiKey = process.env.AI_GATEWAY_API_KEY;
 
const response = await fetch('https://ai-gateway.vercel.sh/v1/responses', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    Authorization: `Bearer ${apiKey}`,
  },
  body: JSON.stringify({
    model: 'anthropic/claude-sonnet-5',
    input: [
      {
        type: 'message',
        role: 'user',
        content: 'What makes a great cup of coffee?',
      },
    ],
    providerOptions: {
      gateway: {
        only: ['anthropic', 'bedrock'],
      },
    },
  }),
});
 
if (!response.ok) throw new Error(await response.text());
console.log(await response.json());
request.py
import json
import os
import urllib.request
 
request = urllib.request.Request(
    'https://ai-gateway.vercel.sh/v1/responses',
    data=json.dumps({'model': 'anthropic/claude-sonnet-5', 'input': [{'type': 'message', 'role': 'user', 'content': 'What makes a great cup of coffee?'}], 'providerOptions': {'gateway': {'only': ['anthropic', 'bedrock']}}}).encode(),
    headers={'Content-Type': 'application/json', 'Authorization': "Bearer " + os.environ["AI_GATEWAY_API_KEY"]},
)
with urllib.request.urlopen(request) as response:
    print(json.load(response))
request.sh
curl --fail-with-body https://ai-gateway.vercel.sh/v1/responses \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "anthropic/claude-sonnet-5",
  "input": [
    {
      "type": "message",
      "role": "user",
      "content": "What makes a great cup of coffee?"
    }
  ],
  "providerOptions": {
    "gateway": {
      "only": [
        "anthropic",
        "bedrock"
      ]
    }
  }
}'

Provider timeouts

You can set per-provider timeouts for BYOK credentials to trigger fast failover when a provider is slow to respond. Pass providerTimeouts in providerOptions.gateway:

"providerOptions": {
  "gateway": {
    "providerTimeouts": {
      "byok": { "anthropic": 3000, "bedrock": 5000 }
    }
  }
}

For full details, limits, and response metadata, see Provider Timeouts.

Automatic caching

Use caching: 'auto' in the request body to let AI Gateway automatically add cache markers for providers that require them (like Anthropic). For full details, supported providers, and examples, see Automatic Caching.

Last updated September 8, 2026
Previous
Images
Next
Python

Was this helpful?

supported.