Most LiteLLM vs. OpenRouter evaluations are framed as feature bake-offs. Both tools present an OpenAI-compatible API, both fan out across dozens of providers, and both retry failed requests, so the matrix makes them look interchangeable. They're architecturally opposite. LiteLLM is software your team deploys and operates. OpenRouter is a service your team calls, and that one difference sets the latency path, the cost model, the compliance posture, and who carries the pager. This comparison covers the seven dimensions where the two diverge, which one fits a given set of constraints, and how teams run both together. **Key takeaways:** - LiteLLM is a self-hosted proxy your team runs. OpenRouter is a managed routing service your team authenticates against, and that split drives every other tradeoff here. - Both vendors publish self-measured latency figures. LiteLLM's runs against a mock upstream, in a configuration its own docs call a benchmark rather than a production deployment. - Neither tool marks up inference. LiteLLM shifts cost into infrastructure and engineering time, while OpenRouter charges a fee when you purchase credits. - Teams whose request data can't leave their network boundary land on LiteLLM before latency or catalog size enters the conversation. - The two compose. LiteLLM treats OpenRouter as an upstream provider through the `openrouter/` model prefix, so the choice isn't always exclusive. ## [Copy link to heading](#litellm-vs-openrouter-at-a-glance)LiteLLM vs OpenRouter at a glance The forwarding itself is the same on both sides. A request arrives, gets matched to a provider, and goes out. What differs is everything around that forward, including which network the request crosses, who holds the provider credentials, where the spend data lands, and who gets paged when the forward fails. Running LiteLLM means deploying it, scaling it, patching it, running its database migrations, and answering the page when it fails. Calling OpenRouter means accepting a public internet hop, a third party on your data path, and their incidents as part of yours. ### [Copy link to heading](#what-this-comparison-is-based-on)What this comparison is based on Every claim here traces to each tool's current official documentation and public issue tracker. Vendor docs are useful but partial by design, and both projects release frequently, so anything version-specific is worth re-checking before a decision goes final. Here's where the two tools split across those seven dimensions: | Dimension | LiteLLM (self-hosted proxy) | OpenRouter (managed service) | | --- | --- | --- | | Hosting model | Runs in your own virtual private cloud (VPC) or cluster | Managed service you authenticate against over the public internet | | Latency path | No external hop, so requests reach providers directly | One added hop on every request, ahead of the provider call | | Provider coverage | More than 100 providers behind one API | A larger catalog, spanning hundreds of models across dozens of providers | | Failover and routing | `simple-shuffle`, `least-busy`, latency-based, and cost-based strategies, with typed fallback categories and health-check-driven pool removal | Price-weighted load balancing by default, skipping providers with recent outages, plus sorting by price, throughput, or latency and user-defined model fallback lists | | Observability | Native push to Prometheus, Datadog, OpenTelemetry, and others, though per-key and per-team routing to Langfuse, LangSmith, and Arize is an Enterprise feature | Trace forwarding to Langfuse through request metadata, with no native server-side metrics push | | Cost model | Free under the MIT license, with governance features behind an Enterprise license. No inference markup, and you pay providers directly | No per-token markup. A fee applies when purchasing credits, and bring your own key (BYOK) usage carries a fee above a plan-dependent allowance | | Ops burden | Schema migrations on every upgrade, PostgreSQL, Redis at multi-instance scale, container orchestration, and on-call ownership | Nothing to host, with availability tied to OpenRouter's own uptime | Hosting model sets the latency path, ops burden follows from it, and the cost model is a consequence of both. ### [Copy link to heading](#what-the-published-numbers-leave-out)What the published numbers leave out LiteLLM's [published benchmarks](https://docs.litellm.ai/docs/benchmarks) measure 2ms median and 13ms P99 proxy overhead at 1,170 requests per second (RPS) across four instances. The run used a mock upstream endpoint, a single PostgreSQL instance, and no Redis, a configuration its own docs flag as a benchmark rather than a production deployment. Neither vendor's figure moves a user-facing number much, because the model call itself runs in the hundreds of milliseconds. The same gap shows up in failover. OpenRouter's [default strategy](https://openrouter.ai/docs/guides/routing/provider-selection) load balances by price, after first excluding any provider that has seen significant outages in the last 30 seconds. LiteLLM hands you the strategy, the fallback categories, and the health-check thresholds, and expects you to configure them. ### [Copy link to heading](#what-self-hosted-operation-costs)What self-hosted operation costs Running LiteLLM in production means running a small platform. Every upgrade brings schema migrations, Redis becomes a requirement at multi-instance scale, and two secrets, `LITELLM_MASTER_KEY` and `LITELLM_SALT_KEY`. Rotate `LITELLM_MASTER_KEY` using LiteLLM’s documented rotation flow. Do not change `LITELLM_SALT_KEY` after storing model credentials, because doing so makes them unreadable. Multiple versions have shipped memory leaks that force restarts, PostgreSQL connection exhaustion appears under gunicorn worker recycling, and the monthly [budget reset job](https://github.com/BerriAI/litellm/issues/13210) hit an out-of-memory crash at roughly 250,000 keys. Observability splits the same way. LiteLLM pushes metrics natively to Prometheus, Datadog, OpenTelemetry, and more than a dozen other backends, with per-key and per-team routing to tracing platforms behind the Enterprise license. OpenRouter forwards traces to Langfuse through request metadata and leaves the rest to your application layer. ## [Copy link to heading](#diving-into-litellm-for-platform-teams-with-residency-requirements)Diving into LiteLLM for platform teams with residency requirements LiteLLM is an open-source proxy under the MIT license that presents one OpenAI-compatible API to your clients while routing to more than 100 upstream providers. It ships as either a monolithic process or a microservices split with separate gateway, backend, and UI components. Production deployments need PostgreSQL for key management and spend tracking, plus Redis for rate limiting once you run multiple instances. An Enterprise tier adds single sign-on (SSO), JSON Web Token (JWT) auth, secrets pulled from AWS Secrets Manager or Vault, audit logs, and a multi-region control plane. ### [Copy link to heading](#litellm-pros-and-cons)LiteLLM pros and cons The case for LiteLLM rests on the proxy sitting inside a network the team already controls: - **No intermediary on the data path:** Requests go from your network straight to the provider, which is what auditable network controls tend to require. - **Deep routing control:** Strategy, fallback categories, and health-check behavior are all yours to configure. Nothing about how a request gets placed is hidden behind someone else's default. - **Native metrics push:** Prometheus, Datadog, and OpenTelemetry integration land without an application-layer shim. - **Per-key and per-team budgets:** The virtual-key model supports tenant-level spend controls, which adapts well to multi-tenant platforms. The same placement generates work that recurs whether or not traffic does: - **It's a platform to operate:** Migrations, a database, a cache, orchestration, and on-call all transfer to your team. - **Governance sits behind a license:** SSO, JWT auth, audit logs, and key rotation aren't in the open-source core. A team adopting the free tier for a compliance requirement can find that the compliance features are the paid ones. - **Documented stability issues under load:** Memory leaks and connection exhaustion have recurred across versions. - **You own the supply chain:** Pinning, verifying, and patching releases becomes your responsibility. **Best for:** Platform teams with existing infrastructure capacity and a hard requirement that request data stay inside a network boundary. Health-data isolation, European residency rules, and audit obligations all point here, as do stacks already running [self-hosted models](/blog/self-hosted-llm) and multi-tenant products that need per-tenant budget enforcement at the routing layer. **Pricing:** The gateway is free to self-host under the MIT license, with no markup on inference, since you hold the provider relationships directly. The cost is the infrastructure underneath it and the engineering time to keep that patched, and both bills arrive in a quiet month too. Governance features require an Enterprise license. ### [Copy link to heading](#where-litellm-stands-out)Where LiteLLM stands out LiteLLM pulls ahead when the requirement is a boundary rather than a feature. When a compliance reviewer asks where a prompt went, "into the VPC and out to the provider" is an answer a team can defend with network controls. No managed service produces the same answer without a contractual layer standing in for a technical one. The advantage only materializes with the platform engineering capacity to back it. A year in, the teams are still happy with their choice and share one trait, which is that someone already owned a database, a cache, and a metrics stack before the gateway showed up. ## [Copy link to heading](#exploring-openrouter-for-teams-without-platform-engineering-capacity)Exploring OpenRouter for teams without platform engineering capacity OpenRouter is a managed service with a single OpenAI-compatible endpoint that routes to a catalog spanning hundreds of models across dozens of providers, with automatic provider failover and nothing to host. Billing is credit-based, so you prepay in US dollars. Its [Auto Router](https://openrouter.ai/docs/guides/routing/routers/auto-router) selects a model per request by classifying the prompt into one of roughly 30 task types. It then ranks models by what the OpenRouter community spent on that task type over a trailing seven-day window. A `cost_tier` setting constrains the price band, and the router degrades to a default model set if classification or rankings are unavailable. ### [Copy link to heading](#openrouter-pros-and-cons)OpenRouter pros and cons Every advantage OpenRouter offers follows from a team not operating the routing layer: - **Nothing to operate:** No database, no cache, no migrations, and no on-call rotation for the routing layer. - **Catalog breadth:** One key and one bill reach a wider model selection than any self-hosted option assembles by default. New models tend to appear here before they appear in a provider list you maintain yourself. - **Failover on by default:** Provider health factors into routing without configuration. - **Fast model swaps:** Changing models is a slug change, which suits prototyping, where the model is still an open question. The same arrangement removes a team's say in how that layer behaves during an incident: - **A third party on the data path:** Every request leaves your network before it reaches a provider. - **Their availability is your availability:** A managed routing layer has its own dependencies and its own incident history, and there's no mechanism to route around it when it degrades. - **Thinner native observability:** Server-side metrics push to Prometheus or Datadog isn't there. Getting spend and latency into an existing dashboard becomes an application-layer job. - **Retention controls have edges:** Zero Data Retention enforcement covers provider routing for inference, and not plugins or tools such as web search. **Best for:** Teams shipping AI features without platform engineering capacity to spare, and teams in an evaluation phase where the model changes weekly. Prototyping is the clearest fit, because the cost of a wrong model choice stays near zero when switching is a string change. **Pricing:** OpenRouter doesn't mark up inference, and you pay the same per-token rates you'd pay a provider directly. Revenue comes from a fee applied when you purchase credits, and BYOK usage carries a fee above a plan-dependent free allowance measured by list-price inference cost. The percentages change from time to time, so the structure is the part worth planning around. You're trading a variable transaction fee for zero fixed infrastructure cost. ### [Copy link to heading](#where-openrouter-stands-out)Where OpenRouter stands out OpenRouter wins the case where a team's binding constraint is attention rather than money. A four-person team shipping a product has a fixed number of engineering hours, and spending them on PostgreSQL connection tuning for a proxy is a choice with a visible opportunity cost. The catalog compounds that. When a new model lands, the question is usually whether it beats the incumbent on one specific workload. Testing it without a new SDK, a new key, and a new billing relationship is the difference between running that experiment and skipping it. ## [Copy link to heading](#which-services-provide-a-single-endpoint-for-multiple-ai-model-providers)Which services provide a single endpoint for multiple AI model providers? Several services expose one API endpoint that routes to multiple AI model providers, including LiteLLM, OpenRouter, and Vercel [AI Gateway](https://vercel.com/docs/ai-gateway). They divide into two structural types: self-hosted proxies, where your team operates the endpoint, and managed routing services, where a third party operates it. Which of the two types a service belongs to is the first filter to apply to any gateway shortlist. ### [Copy link to heading](#where-litellm-and-openrouter-sit-among-single-endpoint-gateways)Where LiteLLM and OpenRouter sit among single-endpoint gateways LiteLLM is the reference implementation of the self-hosted category, and OpenRouter is among the broadest of the managed ones. Neither is the only option in its category, and the shortlist looks different depending on which category survives your residency and staffing constraints. Three comparisons cover the rest of each category: - **Self-hosted options beyond LiteLLM:** Our [open source gateways](https://vercel.com/i/open-source-ai-gateways) comparison covers the projects that compete with it directly. - **Managed options beyond OpenRouter:** Our [OpenRouter alternatives guide](https://vercel.com/i/openrouter-alternatives) covers the managed field and where each one's tradeoffs land. - **The branded head-to-head:** Our [AI Gateway comparison](https://vercel.com/i/vercel-ai-gateway-vs-openrouter) covers that matchup feature by feature. Both categories ask a team to give something up, either ownership of the infrastructure or a clean network boundary. That tradeoff isn't always forced. ## [Copy link to heading](#how-vercel-resolves-the-litellm-and-openrouter-tradeoff-for-production-teams)How Vercel resolves the LiteLLM and OpenRouter tradeoff for production teams For teams already deploying on Vercel, AI Gateway runs on our own [infrastructure](/blog/self-driving-infrastructure) across 20 compute regions, so there's no proxy to operate and nothing on the request path leaving the network. The same operational patterns show up for almost every team running multi-provider traffic in production, and they tend to arrive after launch rather than during evaluation. ### [Copy link to heading](#a-provider-outage-becomes-your-outage)A provider outage becomes your outage A provider incident becomes an application incident without warning. Teams running direct provider calls discover this the first time a model endpoint starts returning 503s and there's no path around it. The retry logic that was supposed to handle it is usually still unwritten. Automatic cross-provider fallback moves that logic out of application code. Across seven months of production traffic through April 2026, AI Gateway's fallback routing [rescued 3.5%](/blog/ai-gateway-production-index) of all requests and 5.1% of all tokens before any team wrote retry logic. Cline ran a [one-week A/B test](/blog/cline-on-ai-gateway) against its previous router and measured a 43.8% drop in API error rates, with P99 streaming latency improving by 10 to 14%. ### [Copy link to heading](#observability-requires-a-license-or-a-metrics-stack)Observability requires a license or a metrics stack Spend and latency data tends to live somewhere other than where the team looks. Self-hosted gateways answer this with a metrics stack the team runs, and managed services answer it with an application-layer shim, so either way, instrumentation becomes a project. Every request routed through AI Gateway gets gateway-layer observability with no instrumentation code, covering requests by model, time to first token, input and output token counts, per-request cost, and which provider served each request. Budgets and quotas apply at the team, project, and API key level from the same surface. ### [Copy link to heading](#compliance-review-blocks-the-launch)Compliance review blocks the launch Nothing stops a launch faster than a compliance question nobody can answer. Once a request leaves the network on a direct call, retention is whatever the provider's default policy says, and a regulated workload has no way to prove otherwise. AI Gateway answers this at the routing layer. Zero Data Retention deletes prompts and outputs after inference and routes only to providers under a ZDR agreement, configurable per request or team-wide. A provider allowlist restricts a team to approved providers on every request without a code change. That covers retention and provider scope, not network topology. A team under a hard requirement that no request leave its own network boundary is still in LiteLLM's column, because AI Gateway is a managed service like the other managed services here. The gateway removes the operational cost of self-hosting, and it doesn't remove the boundary itself. ### [Copy link to heading](#the-proxy-itself-becomes-a-platform-to-run)The proxy itself becomes a platform to run The proxy is its own operational surface. A self-hosted one is a database, a cache, a migration path, and a rotation, and that's before it serves a single production request. After moving its routing to AI Gateway, Zo Computer's [retry rate fell](https://vercel.com/customers/how-zo-computer-improved-ai-reliability-20x-on-vercel) from 7.5% to 0.34%, a 20x reliability improvement that came from moving retries, fallbacks, and provider health monitoring out of its own code. Teams already invested in LiteLLM don't have to choose between the two. LiteLLM is a [supported framework integration](https://vercel.com/docs/ai-gateway/ecosystem/framework-integrations/litellm), so an existing LiteLLM deployment can route through AI Gateway with a model prefix and a key: ``` import litellm response = litellm.completion( model="vercel_ai_gateway/openai/gpt-5.6-sol", messages=[{"role": "user", "content": "Summarize this paragraph."}], ) ``` Inside a Vercel deployment, `VERCEL_OIDC_TOKEN` is provided automatically, so no provider key has to live in application code. ## [Copy link to heading](#settle-who-runs-the-proxy,-then-ship)Settle who runs the proxy, then ship Every dimension that separates these two tools is downstream of a single question, and it's the one most evaluations answer last. Settle who operates the endpoint, and most of the matrix resolves itself. A residency requirement points at a self-hosted proxy regardless of catalog size, and a team without infrastructure capacity shouldn't be running one, regardless of how good the routing configuration looks on paper. The question worth asking is which failure mode your team is staffed to absorb. For teams building on Vercel, that choice comes with a third option: - **Unified provider access:** One endpoint and one key reach hundreds of models, with model swaps as a config change rather than a new SDK. - **Automatic cross-provider fallback:** Provider outages become routing decisions instead of user-facing errors, with no retry logic in application code. - **Gateway-layer observability:** Per-request cost, token counts, latency, and provider attribution arrive without instrumentation code. - **Zero Data Retention and provider allowlists:** Retention and provider scope are enforced on every request, configurable per request or across a team. - **Budgets and key management:** Spend caps and API keys apply at the team, project, and key level from the dashboard, CLI, or API. Start a project at [vercel.com/new](https://vercel.com/new), or browse [Vercel templates](https://vercel.com/templates) to route your first request through it. ## [Copy link to heading](#frequently-asked-questions-about-litellm-vs-openrouter)Frequently asked questions about LiteLLM vs OpenRouter ### [Copy link to heading](#can-you-use-litellm-and-openrouter-together)Can you use LiteLLM and OpenRouter together? Yes, and it's a common production pattern. Adding the `openrouter/` model prefix points a LiteLLM upstream at OpenRouter, putting local virtual keys, role-based access control (RBAC), and budget enforcement in front of the wider catalog. You pay both costs, running LiteLLM and accepting the hop. ### [Copy link to heading](#what-is-the-latency-difference-between-litellm-and-openrouter)What is the latency difference between LiteLLM and OpenRouter? LiteLLM adds no external hop, since it runs inside your network and calls providers directly. OpenRouter adds one public internet hop per request. Both vendors publish self-measured overhead figures under favorable conditions, and against a 500ms to 3s LLM call, that difference rarely decides anything. ### [Copy link to heading](#does-openrouter-store-your-prompts)Does OpenRouter store your prompts? You can prevent it. Zero Data Retention can be enforced globally, per model group, per guardrail, or per request, restricting routing to endpoints that don't retain data. The setting governs inference routing only, so anything running alongside it carries its own retention policy. ### [Copy link to heading](#is-litellm-free-to-run-in-production)Is LiteLLM free to run in production? Self-hosting the proxy costs nothing under its MIT license, though reliable production operation is not free. Compute across multiple instances, PostgreSQL, Redis, and engineering maintenance all recur, and governance features, including SSO, JWT auth, and audit logs, require an Enterprise license.