An OpenAI-compatible API lets an app use OpenAI client libraries to call models from other providers. Teams switch to try a model that handles a task better, compare models on the same prompts, or keep a second provider ready for outages. For supported Chat Completions requests, the first step is often to change the base URL, API key, and model name.
Compatibility covers request and response formats, but it does not guarantee identical behavior. Check which features and authentication methods the provider supports. Test streaming, structured output, reasoning settings, and errors before production. A successful first request won't reveal every difference.
Copy link to headingWhat is an OpenAI-compatible API?
For Chat Completions, an OpenAI-compatible API accepts supported requests in OpenAI's format and returns responses in a similar shape. Each request carries a list of messages and typically authenticates with a bearer token in the Authorization header. OpenAI's SDKs can call the compatible endpoints that providers publish. Some providers also support other OpenAI API formats, such as Responses or embeddings.
Model providers offer compatibility layers so developers can try their models without adopting a new software development kit (SDK). Gateways use the same format to accept requests for multiple providers at one URL, as with AI Gateway's compatible endpoints.
Copy link to headingHow does an OpenAI-compatible API differ from OpenAI's own API?
Check what happens when a request includes a field the provider doesn't support. Anthropic's layer ignores most unsupported fields instead of rejecting them. Together AI accepts service_tier, store, metadata, and prediction but ignores their values. Compare these behaviors before switching providers:
Claude's compatibility layer moves every system and developer message to the start of the conversation and joins them into one. An app that adds instructions midway through a conversation needs the native Claude API to keep them in place.
Copy link to headingWhy does an OpenAI-compatible API matter?
A provider or gateway that accepts this format can work with OpenAI's SDKs and the tools built on them. It must support the endpoints and features those clients use. That lets developers reuse client code in several ways:
Configuration instead of code: The official OpenAI Python library reads its base URL from the
OPENAI_BASE_URLenvironment variable when the app supplies no explicit base URL. If the app also reads its key and model from configuration, switching providers may need no client code changes.Model comparison before commitment: Anthropic built its layer for testing and comparing models. Teams can run existing prompts against Claude before moving to the native API.
Coding agents on supported models: Hermes, a coding agent, connects to supported, tool-capable language models through AI Gateway's Chat Completions API.
Support for clients that extend the format: Cursor sends some request bodies that the standard OpenAI schema rejects, so AI Gateway normalizes them on a dedicated endpoint.
A gateway handles some compatibility differences between providers. When calling providers directly, the app must handle any differences its SDKs don't cover.
Copy link to headingCore components of an OpenAI-compatible API
An app can send requests directly to a provider or through a large language model (LLM) proxy. In either case, check the connection settings, request and response formats, and supported endpoints.
Copy link to headingThe settings you swap
Include the provider's required path prefix in baseURL. The OpenAI Node SDK appends /chat/completions to that base, so a provider that serves under /v1 needs that prefix in the URL. Check model IDs too. Together AI, for example, returns a 404 for unsupported OpenAI model strings such as gpt-4o.
This example calls AI Gateway with the official openai package:
import OpenAI from 'openai';
const client = new OpenAI({ apiKey: process.env.AI_GATEWAY_API_KEY, baseURL: 'https://ai-gateway.vercel.sh/v1',});
const completion = await client.chat.completions.create({ model: 'anthropic/claude-opus-5', messages: [{ role: 'user', content: 'Summarize this support ticket in one sentence: I was charged twice for my subscription renewal and need a refund for the duplicate charge.' }],});
console.log(completion.choices[0].message.content);The key authorizes requests on your account. Run this code on the server with AI_GATEWAY_API_KEY set. AI Gateway uses provider/model strings, while other providers document their own model IDs. Keep model names in configuration if the app needs to switch between them.
Copy link to headingThe request and response contract
A Chat Completions request contains messages, and its response contains choices. A non-streamed response is a chat.completion object with an ID, the model used, and a choices array of complete messages. A streamed response arrives as a series of chat.completion.chunk objects. The client assembles partial delta fields and handles any usage-only chunks separately.
Handle the two modes separately. One reads whole messages, and the other assembles deltas. Both can work across providers that follow the format, but streaming needs its own compatibility tests.
Copy link to headingThe endpoint surface beyond chat
A provider's compatible endpoints may expose only part of its native API. Check for missing features within chat and for missing endpoints. Current versions of OpenAI's terminal coding agent, Codex, use Responses rather than Chat Completions. AI Gateway provides a dedicated Codex compatibility endpoint at /codex/v1.
AI Gateway maps the root-level embedding dimensions field to each provider's expected field. Clients can use the OpenAI parameter where the selected model supports it. Before choosing a provider, list every endpoint and field your app calls and check each one against the provider's compatibility page.
Copy link to heading5 best practices for swapping OpenAI-compatible API providers
A basic request test can miss differences that cause failures in production. Handle those differences in application code, an SDK, or an LLM gateway.
Copy link to headingTest streamed tool calls chunk by chunk
An agent loop that passes non-streamed tool-call tests can stall when streaming. A streamed Chat Completions tool call arrives in fragments keyed by index. Initial fragments identify the call and function. Later fragments can carry argument text without repeating the id and function name. Those fields may be absent or null. A parser that copies every field from every fragment can overwrite a stored value. Check the Chat Completions streaming schema.
Group fragments by index and append the argument text. Keep stored identifiers when later fragments omit them. Run this test with streaming on against each candidate provider to check that it follows the format.
Copy link to headingValidate structured output in your own code
A response can contain valid JSON without matching your schema, or it can fail to parse JSON. Test tool-argument validation separately from structured response output. Anthropic's layer ignores the strict flag on function definitions. DeepSeek accepts only text and json_object as Chat Completions response formats. Groq doesn't support streaming or tool use alongside its Structured Outputs.
Parse the output and validate it against the schema in code. If either check fails, retry within a set limit or return an explicit error. Validate tool arguments before executing a tool.
Copy link to headingMap reasoning controls per model
Support for OpenAI's reasoning_effort varies by provider. Switching models can change reasoning behavior and request cost. Anthropic's layer ignores reasoning_effort and accepts a separate thinking object, which OpenAI Python clients send through extra_body. Gemini maps reasoning_effort to thinking_level or thinking_budget, depending on the model. Even within OpenAI, GPT-6 Astra does not support reasoning_effort: 'none'.
Store each model's reasoning settings in configuration so the calling code doesn't need to handle those differences. Test each entry against the exact model ID you plan to use.
Copy link to headingRead usage from the final stream chunk
Handle streamed usage data separately from content. With stream_options.include_usage: true, OpenAI streams one extra chunk before data: [DONE]. It carries usage for the whole request and an empty choices array. Code that reads choices[0] on every chunk can fail here. An interrupted stream may never deliver the usage chunk.
Check that choices contains an entry before reading it. Record missing usage as unknown rather than zero to avoid understating costs when a stream ends early.
Copy link to headingNormalize provider errors and rate limit headers
Providers use different error codes and headers, so retry logic written for OpenAI can misclassify their failures. Anthropic returns HTTP 529 with overloaded_error when its API is overloaded. On Groq, x-ratelimit-limit-requests counts requests per day, while retry-after appears only on a 429. Together AI returns OpenAI-shaped error bodies with its own type and code values.
Start with the HTTP status, then inspect the provider's error body and retry headers. Decide whether to retry, wait longer, or change the request or account settings. Limit retries and use backoff where a retry can succeed.
Copy link to headingHow Vercel powers OpenAI-compatible APIs in production
With an AI gateway, the client uses one base URL and credential. The gateway handles routing and some failures. Switching providers can then be a configuration change, though model behavior still needs testing. Tailscale moved its whole company to AI Gateway in seconds without employees noticing the change.
Copy link to headingReach every provider through one endpoint with AI Gateway
Each additional provider brings its own credentials, model IDs, and routing settings. AI Gateway provides an OpenAI-compatible Chat Completions API at https://ai-gateway.vercel.sh/v1 on every Vercel plan. Change the base URL, key, and model ID to send supported requests through it. Model IDs use provider/model strings. Free credits cover a subset of models; other models require purchased credits.
AI Gateway also accepts the OpenAI Responses API, Anthropic Messages, and OpenResponses formats. Requests authenticate with an AI Gateway API key or with a Vercel OpenID Connect (OIDC) token on Vercel deployments.
Copy link to headingRoute around provider outages with AI Gateway
A client that calls one provider directly needs another way to route requests during an outage. AI Gateway can try other eligible providers that serve the same model. A models array in providerOptions.gateway lists backup models to try in order if the primary model's eligible providers fail. Vercel's May 2026 production index reported that roughly 3.5% of AI Gateway requests completed after a fallback. Zo Computer reported a retry-rate reduction from 7.5% to 0.34% after moving retries and routing to AI Gateway.
The order setting lists preferred providers in sequence. AI Gateway can still try unlisted eligible providers afterward. The only setting restricts routing to the listed providers, so a request fails when none can serve it. Both fields belong under providerOptions.gateway. In the OpenAI Python SDK, send that object through extra_body.
Copy link to headingKeep one reasoning format across model families with AI Gateway
Providers use different reasoning formats, so switching a tool-calling loop between model families can require changes to how the app stores and replays reasoning. AI Gateway normalizes reasoning from supported models into a shared structure. When reasoning data is available, the reasoning_details array preserves provider-specific details alongside the reasoning text. Streaming responses use the corresponding delta fields.
Set the effort field in a reasoning object to control reasoning across supported models. AI Gateway translates it for each model and provider. The request schema accepts levels from none to max. Check each model's catalog entry for the levels it supports. The object is an AI Gateway extension to Chat Completions, so OpenAI Python clients send it through extra_body. Clients that only need effort can also use the standard reasoning_effort field.
Copy link to headingHold sensitive prompts to zero data retention providers with AI Gateway
Changing providers changes who receives the prompt. Some compatibility layers accept and ignore the request's store field, so it cannot establish the provider's retention policy. With zero data retention (ZDR) enabled, AI Gateway routes requests only to eligible ZDR-compliant providers, including fallbacks. Per-request and team-wide ZDR are available to Pro and Enterprise teams and teams that purchase AI Gateway credits. An active Token Pass also enables per-request ZDR.
For per-request ZDR, set zeroDataRetention: true under providerOptions.gateway. An OpenAI Python client sends this through extra_body. Per-request ZDR has no additional charge. Enable team-wide ZDR in the dashboard to apply it to every request for that team. It costs $0.10 per 1,000 successful requests that return usage data. AI Gateway excludes bring-your-own-key (BYOK) provider keys from ZDR routing by default unless you mark them ZDR-compliant.
Copy link to headingTrace and limit spend per request with AI Gateway budgets
Calling providers directly can split spending records across their dashboards. AI Gateway records request status, provider, latency, token usage, cost, and routing attempts in one place. Budgets reject new requests once an applicable team, project, API key, or member budget is exhausted. BYOK spending does not count toward budget limits, but an exceeded budget still blocks BYOK requests within its scope. AI Gateway checks budgets at the start of each request. A request that crosses the limit can still be completed and bring total spending above it.
An exceeded budget returns HTTP 402 with the error type quota_for_entity_exceeded. Requests remain blocked until the budget resets, or someone raises or removes the limit. A 429 may come from AI Gateway or the upstream provider. Whether a retry can succeed depends on its cause. Inspect the status, error body, and retry headers before trying again.
Copy link to headingTest the behavior, then swap the base URL
The OpenAI format can reduce client code changes when switching providers. Behavior differences still need testing. The tests depend on how the app uses each response. A chat reply may tolerate differences that would break a tool-calling agent or a JSON-parsing pipeline.
AI Gateway provides:
One OpenAI-compatible endpoint for supported providers: OpenAI clients can use the same integration for supported endpoints and models across providers.
Model and provider fallbacks per request: AI Gateway tries eligible backup providers and configured fallback models. It returns an error if no route succeeds.
Normalized reasoning across supported model families: A shared format reduces changes to conversation-management code and preserves provider-specific reasoning details.
Zero data retention routing for eligible accounts: Apply retention rules to individual requests or the whole team. Eligibility and pricing depend on the option used.
Per-request cost logs with budgets: Track provider usage in one place and reject new requests after an applicable spending limit is exhausted.
Start a new project to route your first request through AI Gateway, or browse templates to begin from a working AI app.
Copy link to headingFrequently asked questions about OpenAI-compatible APIs
Copy link to headingIs Claude or Gemini OpenAI-compatible?
Yes, with gaps. Anthropic and Google both publish OpenAI-compatible endpoints that cover less than their native APIs. Anthropic's layer rejects audio and file inputs with a 400 error and leaves prompt caching to the native Claude API. Gemini's layer adds image generation and accepts Gemini-only options through extra_body in the OpenAI Python SDK.
Copy link to headingIs an OpenAI-compatible API the same as the Responses API?
No. Responses is a separate OpenAI API, and a provider that supports Chat Completions may not support it. GPT-6 Astra and GPT-6.1 Sol require Responses for tool calling. Ollama, for example, serves /v1/responses for stateless requests, without previous_response_id or conversation support.
Copy link to headingDoes an OpenAI-compatible API cover every OpenAI endpoint?
Not always. Compatibility often covers inference endpoints only. Together AI, for example, does not implement OpenAI's Assistants, Threads, Runs, or moderations.create interfaces. Its native Batch and Files APIs sit outside the OpenAI-compatible /v1 endpoints. Gemini's compatible Batch interface supports job creation, status checks, and results, but file uploads and downloads require the native SDK. Check batch and administrative endpoints separately before a migration.