--- title: Keep AI responses available with model fallbacks description: Configure AI Gateway fallbacks for an existing generation flow, checking model compatibility, error handling, and retry boundaries. url: "https://vercel.com/kb/prompts/add-ai-gateway-model-fallbacks" last_updated: 2026-11-07 related_resources: "[Configure model fallbacks](https://vercel.com/docs/ai-gateway/models-and-providers/model-fallbacks)": "Choose alternative models for failed requests and inspect which model and provider handled the response." "[Provider Routing and Fallbacks](https://vercel.com/docs/ai-gateway/models-and-providers/provider-options)": "Control which providers serve model requests and how AI Gateway orders providers and handles failover." --- # Keep AI responses available with model fallbacks Configure AI Gateway fallbacks for an existing generation flow, checking model compatibility, error handling, and retry boundaries. Give an existing AI generation flow a backup model when its primary model fails. This prompt adds a bounded fallback configuration through AI Gateway while preserving the app’s response contract. Open or import your project and identify the generation endpoint to update. Have your permitted models and data-handling requirements available. The handoff sends the prompt text only, without repository context or AI Gateway credentials. Expect explicit fallback configuration, useful failure states, and simulated outage tests. Switching models can change output quality, cost, and latency, so this work also records compatibility assumptions that need review before release. ## Agent prompt ```text Add AI Gateway model fallbacks to one existing generation flow so it can use a compatible backup when the primary model fails. Read repository instructions and inspect the endpoint, package manager, installed SDK, model configuration, streaming, tool calls, and current retry policy. If the intended project is unavailable, ask me to open or import it. Verify the installed SDK and current AI Gateway docs before changing configuration. Reuse the existing integration and avoid a broader provider or framework migration. Identify the required response format, context size, modalities, tool support, data-handling restrictions, and acceptable cost range. Use existing approved models; if no compatible backup is defined, ask for that choice before making live requests. Do not invent model IDs. Configure a short ordered fallback list on the server. Distinguish changing models from routing the same model through another provider: a provider preference is not an allowlist. Preserve any existing provider restrictions and retention requirements across every fallback. Keep the response contract stable. Validate structured output before downstream use and retain existing authorization, input limits, and error redaction. Make missing configuration actionable. Preserve cancellation and a bounded request deadline, and account for SDK retries and Gateway attempts together so they do not multiply unnoticed. Handle failure after streaming begins explicitly; do not silently concatenate answers from different models. Never replay a whole agent turn after a tool may have changed external state. Preserve tool idempotency and approvals, and surface uncertain outcomes for reconciliation. Record available model and attempt metadata without logging prompts, credentials, or private payloads; do not infer a successful fallback from the configured list alone. Use the existing test tools with mocked Gateway responses to cover primary success, fallback success, all attempts failing, incompatible structured output, cancellation, and a stream that fails after partial output. Assert that repeated attempts cannot duplicate a side-effecting tool. Explain that these fixtures test the application’s handling, not Gateway’s live routing or model quality. Implement locally and ask before paid inference calls, changing account routing policy, or deployment. Report the chosen configuration, compatibility assumptions, commands and results, blocked checks, required environment setup, local run steps, and the steps needed for an approved live smoke test. ``` ## Related resources - [Configure model fallbacks](https://vercel.com/docs/ai-gateway/models-and-providers/model-fallbacks): Choose alternative models for failed requests and inspect which model and provider handled the response. - [Provider Routing and Fallbacks](https://vercel.com/docs/ai-gateway/models-and-providers/provider-options): Control which providers serve model requests and how AI Gateway orders providers and handles failover. ## Try other prompts - [Build an AI chat app with streaming responses](/kb/prompts/build-ai-chat-app): Build a chat interface with the AI SDK and AI Gateway, including streamed responses, cancellation, and recoverable errors. - [Evaluate an agent before changing its prompt](/kb/prompts/evaluate-agent-prompt-changes): Build a repeatable evaluation for an AI SDK agent, compare prompt versions, and surface failures hidden by an average score. ## Reference Architecture Choose a stack and work through the decisions behind it, with curated documentation, guides, templates, and courses to help you build. - [Agent stack](/kb/architecture/agents): Build and operate agents with models, tools, and durable workflows. - [Web applications](/kb/architecture/web): Choose rendering, data, authentication, and caching approaches. - [Backends](/kb/architecture/backends): Choose a runtime, connect services, and run background work. [Browse all stacks](/kb/architecture)