Skip to content
Docs

AI Gateway Reranking

Rerank documents by relevance to a search query. Reranking is useful for improving search results in retrieval-augmented generation (RAG) pipelines by re-scoring candidate documents after an initial retrieval step.

To see which models AI Gateway supports for reranking, use the Reranking filter at the AI Gateway Models page.

Reranking is also available through the Cohere-compatible Cohere Rerank API (/v1/rerank and /v2/rerank), for use with the Cohere SDK or plain HTTP.

Basic usage

For SDK options and result types, see AI SDK reranking and Python reranking.

app/api/rerank/route.ts
import { rerank } from 'ai';
 
export async function GET() {
  const result = await rerank({
    model: 'cohere/rerank-v3.5',
    query: 'What is the capital of France?',
    documents: [
      'Paris is the capital of France.',
      'Berlin is the capital of Germany.',
      'Madrid is the capital of Spain.',
    ],
    topN: 2,
  });
 
  return Response.json(result.ranking);
}
rerank.py
import asyncio
import ai
 
async def main():
    result = await ai.ops.rerank(
        ai.get_model('cohere/rerank-v3.5'),
        ["Paris is the capital of France.", "Berlin is the capital of Germany.", "Madrid is the capital of Spain."],
        'What is the capital of France?',
        params=ai.ops.RerankParams(top_n=2),
    )
    print(result.value)
 
asyncio.run(main())
rerank-rest.ts
const response = await fetch('https://ai-gateway.vercel.sh/v2/rerank', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.AI_GATEWAY_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: 'cohere/rerank-v3.5',
    query: 'What is the capital of France?',
    documents: [
      'Paris is the capital of France.',
      'Berlin is the capital of Germany.',
      'Madrid is the capital of Spain.',
    ],
    top_n: 2,
  }),
});
if (!response.ok) throw new Error(await response.text());
console.log(await response.json());
rerank-rest.py
import json
import os
import urllib.request
 
request = urllib.request.Request(
    "https://ai-gateway.vercel.sh/v2/rerank",
    data=json.dumps({"model": "cohere/rerank-v3.5", "query": "What is the capital of France?", "documents": ["Paris is the capital of France.", "Berlin is the capital of Germany.", "Madrid is the capital of Spain."], "top_n": 2}).encode(),
    headers={
        'Authorization': "Bearer " + os.environ["AI_GATEWAY_API_KEY"],
        'Content-Type': "application/json"
    },
)
with urllib.request.urlopen(request) as response:
    print(json.load(response))
rerank-rest.sh
curl --fail-with-body https://ai-gateway.vercel.sh/v2/rerank \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "cohere/rerank-v3.5",
  "query": "What is the capital of France?",
  "documents": [
    "Paris is the capital of France.",
    "Berlin is the capital of Germany.",
    "Madrid is the capital of Spain."
  ],
  "top_n": 2
}'

The rerank function returns a ranking array sorted by relevance score, along with the rerankedDocuments in order:

// result.ranking
[
  { originalIndex: 0, score: 0.89, document: 'Paris is the capital of France.' },
  { originalIndex: 2, score: 0.15, document: 'Madrid is the capital of Spain.' },
];
 
// result.rerankedDocuments
['Paris is the capital of France.', 'Madrid is the capital of Spain.']

Gateway provider instance

If you're using the Gateway provider instance, specify reranking models with gateway.rerankingModel(...).

app/api/rerank/route.ts
import { rerank } from 'ai';
import { gateway } from '@ai-sdk/gateway';
 
export async function GET() {
  const result = await rerank({
    model: gateway.rerankingModel('cohere/rerank-v3.5'),
    query: 'What is the capital of France?',
    documents: [
      'Paris is the capital of France.',
      'Berlin is the capital of Germany.',
      'Madrid is the capital of Spain.',
    ],
    topN: 2,
  });
 
  return Response.json(result.ranking);
}

Reranking is available through the AI SDK and through the Cohere-compatible /v1/rerank and /v2/rerank REST endpoints. It is not supported through the OpenAI-compatible or Anthropic-compatible endpoints.

Amazon Bedrock reranking requires SigV4 credentials (accessKeyId and secretAccessKey) and does not accept API keys. If you BYOK and plan to use reranking models through Bedrock, you must use SigV4 credentials.

Last updated September 8, 2026
Previous
Embeddings
Next
Inputs & Tools

Was this helpful?

supported.