Skip to content
Docs

AI Gateway Model Filtering

Provider filtering, ordering, and sorting controls which providers serve a request. Model filtering instead constrains routing by a capability of the model itself using has in providerOptions.gateway.

Require model capabilities with has

Use the has array to restrict routing to provider models that have all of the given capabilities. Because a capability is a property of the model rather than the credential, this filter applies to both system and BYOK credentials.

The following capabilities are supported:

CapabilityDescription
freeModels with free inference
implicit-cachingModels that perform automatic (implicit) prompt caching
reasoningModels that support reasoning
structured-outputModels that produce schema-constrained output (JSON schema)
tool-useModels that support tool calling
visionModels that accept image input

AI Gateway sets structured-output on a provider model only after a live probe shows that provider produces schema-constrained output, so the same model can carry the capability on some providers and not others. has: ['structured-output'] routes only to the providers that carry it.

Decision requests don't need structured-output in has. AI Gateway adds the requirement automatically to language-model fallbacks, and adding it to the request-wide has would also filter the native decision model.

These examples use AI SDK 7 and the AI SDK for Python beta. Set AI_GATEWAY_API_KEY before running them. See API format differences for setup, request fields, and response handling.

See the AI SDK capability-filtering reference for SDK configuration and usage.

model-filtering.ts
import { generateText } from 'ai';
 
const { text } = await generateText({
  model: 'deepseek/deepseek-v4-flash',
  prompt: 'Explain quantum computing in two sentences.',
  providerOptions: {
    gateway: {
      has: ['implicit-caching'],
    },
  },
});
 
console.log(text);
model-filtering_ai.py
import asyncio
import ai
 
async def main():
    model = ai.get_model("deepseek/deepseek-v4-flash")
    messages = [ai.user_message("Explain quantum computing in two sentences.")]
    params = ai.InferenceRequestParams(
        extra_body={"providerOptions": {"gateway": {"has": ["implicit-caching"]}}}
    )
    async with ai.stream(model, messages, params=params) as stream:
        async for event in stream:
            if isinstance(event, ai.events.TextDelta):
                print(event.chunk, end="", flush=True)
    print()
 
asyncio.run(main())
model-filtering-chat.ts
import OpenAI from 'openai';
 
const client = new OpenAI({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh/v1',
});
 
const response = await client.chat.completions.create({
  model: 'deepseek/deepseek-v4-flash',
  messages: [
    {
      role: 'user',
      content: 'Explain quantum computing in two sentences.',
    },
  ],
  // AI Gateway extension fields are not included in the upstream SDK types.
  ...{
    providerOptions: {
      gateway: {
        has: ['implicit-caching'],
      },
    },
  },
});
 
console.log(response.choices[0]?.message.content);
model-filtering_chat.py
import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.environ["AI_GATEWAY_API_KEY"],
    base_url="https://ai-gateway.vercel.sh/v1",
)
 
response = client.chat.completions.create(
    model="deepseek/deepseek-v4-flash",
    messages=[{"role": "user", "content": "Explain quantum computing in two sentences."}],
    extra_body={"providerOptions": {"gateway": {"has": ["implicit-caching"]}}},
)
 
print(response.choices[0].message.content)
model-filtering-chat.sh
curl --fail-with-body https://ai-gateway.vercel.sh/v1/chat/completions \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "deepseek/deepseek-v4-flash",
  "messages": [
    {
      "role": "user",
      "content": "Explain quantum computing in two sentences."
    }
  ],
  "providerOptions": {
    "gateway": {
      "has": [
        "implicit-caching"
      ]
    }
  }
}'
model-filtering-messages.ts
import Anthropic from '@anthropic-ai/sdk';
 
const client = new Anthropic({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh',
});
 
const response = await client.messages.create({
  model: 'deepseek/deepseek-v4-flash',
  messages: [
    {
      role: 'user',
      content: 'Explain quantum computing in two sentences.',
    },
  ],
  max_tokens: 1024,
  ...{
    providerOptions: {
      gateway: {
        has: ['implicit-caching'],
      },
    },
  },
});
 
for (const block of response.content) {
  if (block.type === 'text') console.log(block.text);
}
model-filtering_messages.py
import os
from anthropic import Anthropic
 
client = Anthropic(
    api_key=os.environ["AI_GATEWAY_API_KEY"],
    base_url="https://ai-gateway.vercel.sh",
)
 
response = client.messages.create(
    model="deepseek/deepseek-v4-flash",
    messages=[{"role": "user", "content": "Explain quantum computing in two sentences."}],
    max_tokens=1024,
    extra_body={"providerOptions": {"gateway": {"has": ["implicit-caching"]}}},
)
 
for block in response.content:
    if block.type == "text":
        print(block.text)
model-filtering-messages.sh
curl --fail-with-body https://ai-gateway.vercel.sh/v1/messages \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
  "model": "deepseek/deepseek-v4-flash",
  "messages": [
    {
      "role": "user",
      "content": "Explain quantum computing in two sentences."
    }
  ],
  "max_tokens": 1024,
  "providerOptions": {
    "gateway": {
      "has": [
        "implicit-caching"
      ]
    }
  }
}'
model-filtering-responses.ts
import OpenAI from 'openai';
 
const client = new OpenAI({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh/v1',
});
 
const response = await client.responses.create({
  model: 'deepseek/deepseek-v4-flash',
  input: 'Explain quantum computing in two sentences.',
  ...{
    providerOptions: {
      gateway: {
        has: ['implicit-caching'],
      },
    },
  },
});
 
console.log(response.output_text);
model-filtering_responses.py
import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.environ["AI_GATEWAY_API_KEY"],
    base_url="https://ai-gateway.vercel.sh/v1",
)
 
response = client.responses.create(
    model="deepseek/deepseek-v4-flash",
    input="Explain quantum computing in two sentences.",
    extra_body={"providerOptions": {"gateway": {"has": ["implicit-caching"]}}},
)
 
print(response.output_text)
model-filtering-responses.sh
curl --fail-with-body https://ai-gateway.vercel.sh/v1/responses \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "deepseek/deepseek-v4-flash",
  "input": "Explain quantum computing in two sentences.",
  "providerOptions": {
    "gateway": {
      "has": [
        "implicit-caching"
      ]
    }
  }
}'

In this example:

  • Restriction: Only provider models with the implicit-caching capability are eligible for routing and fallbacks. The filter applies to both system and BYOK credentials.
  • All capabilities required: When you list more than one capability, a model must have every one to be eligible.
  • Error on mismatch: If no provider model for the requested model has the capabilities, the request fails. Unsupported values are rejected.

has: ['implicit-caching'] ensures you only route to models that cache automatically. To instead let AI Gateway add cache markers for providers that require explicit caching, see Automatic Caching.

Filter by serving weight format

Weight-format conditions route on the weight format the serving provider reports for the model (for example, fp8 or int4 quantized weights). Use quantization:<value> to require a format, or !quantization:<value> to exclude one:

model-filtering.ts
import { generateText } from 'ai';
 
const { text } = await generateText({
  model: 'deepseek/deepseek-v4-flash',
  prompt: 'Summarize this report',
  providerOptions: {
    gateway: {
      has: ['!quantization:fp8'], // Skip providers serving fp8-quantized weights
    },
  },
});

A negated condition also matches providers with no recorded weight format.

Combining with provider filtering

has composes with the provider-level order, only, and sort options. The model-capability filter and the provider filters are both applied, so the request is routed only to providers that satisfy your provider constraints and whose model has the required capabilities.

Quick reference

OptionTypeDescription
hasArray<'free' | 'implicit-caching' | 'reasoning' | 'structured-output' | 'tool-use' | 'vision' | 'quantization:<value>' | '!quantization:<value>'>Restrict routing to models that have all of the given capabilities or satisfy the given weight-format conditions

Set has under providerOptions.gateway in every request format shown above. OpenAI and Anthropic Python clients pass that object through extra_body; TypeScript clients spread the Gateway extension into the request. See API format differences.

Capability filtering in virtual models

A virtual model can require capabilities for all of its traffic. Its has list replaces the request's, and every capability in it must hold.

Last updated October 2, 2026
Previous
Filtering, Ordering & Sorting
Next
Batch Processing

Was this helpful?

supported.