Searches for LiteLLM alternatives climbed in March 2026, after attackers compromised the security scanner in LiteLLM's build pipeline, took its publishing credentials, and pushed two backdoored releases to PyPI. Two more vulnerabilities in the proxy's own code followed, and all three reached the Cybersecurity and Infrastructure Security Agency (CISA) Known Exploited Vulnerabilities catalog between March and June. Most of the searching that followed assumes self-hosting the proxy was the mistake.

None of that settles whether to keep running the proxy yourself. What it reopens is custody. A self-hosted proxy keeps provider keys and prompts inside your network and hands you the patching, and a managed gateway reverses both.

LiteLLM [deploys as a server](https://vercel.com/changelog/litellm-server-now-supported-on-vercel) on Vercel and [routes through AI Gateway](https://vercel.com/docs/ai-gateway/ecosystem/framework-integrations/litellm) behind a `vercel_ai_gateway/` prefix. The seven options below are compared on license, hosting model, provider coverage, operational footprint, and patch record.

**Key takeaways:**

- Self-hosting costs throughput, added latency, and three stateful services on the on-call rotation, which is the price of keeping keys and prompts inside your network.

- Provider counts do not compare across vendors, and LiteLLM's own figures range from 100+ providers in its docs to 140+ providers and 1,800+ models on its site.

- Two of the seven options here changed owners inside twelve months and one moved to maintenance mode, which makes continuity a sharper filter than feature parity.

- A gateway is the highest-value target in an AI stack, because it holds every provider key and reads every prompt in plaintext.

- The migration cost sits in accumulated observability data, caching rules, and routing configuration, not in the `base_url` swap.

## [Copy link to heading](#what-litellm's-2026-security-record-changes)What LiteLLM's 2026 security record changes

LiteLLM is the most-adopted option here, with 57,000 GitHub stars, production deployments at NVIDIA, Netflix, and Okta, and broader provider coverage than anything else in this comparison, all under an MIT license. Its security year matters to so many teams because of that reach.

Those three listings came from two different kinds of failure. One was an attack on the build pipeline that never touched LiteLLM's code, and the other was a pair of vulnerabilities in the code itself.

### [Copy link to heading](#the-march-supply-chain-compromise)The March supply-chain compromise

LiteLLM's continuous integration pipeline ran Trivy, a security scanner that attackers had compromised between March 19 and 23, 2026. Credentials taken from that pipeline were used to publish two poisoned releases straight to PyPI on March 24, bypassing the project's own release workflow. Versions 1.82.7 and 1.82.8 were live from 10:39 UTC for roughly 40 minutes before PyPI quarantined them. The payload harvested environment variables, SSH keys, cloud credentials, and database passwords from any environment that installed them.

Teams running the official Docker image were unaffected, because that path pins its dependencies in `requirements.txt`. [LiteLLM's response](https://docs.litellm.ai/blog/security-update-march-2026) was substantive: paused releases pending a supply-chain review, SHA-256 checksums published for audited releases, and v1.83.0 shipped through a rebuilt pipeline with isolated environments. [The DSPy maintainers](https://github.com/stanfordnlp/dspy/issues/9500) pinned the dependency at 1.82.6 or lower while they reconsidered it.

### [Copy link to heading](#two-proxy-vulnerabilities-exploited-in-the-wild)Two proxy vulnerabilities exploited in the wild

The supply chain was not the only problem. Two flaws in the proxy's own code reached the KEV catalog within 60 days of each other, both scoring above 8.5 on the Common Vulnerability Scoring System (CVSS):

- **CVE-2026-42208, SQL injection, CVSS 9.3:** An unauthenticated request could read or modify the proxy's database, because the API key check built its query from the caller's own key. A crafted `Authorization` header on any model route was enough. Attacks appeared 36 hours after the advisory, and the KEV listing followed on May 8, 2026.

- **CVE-2026-42271, command injection, CVSS 8.7:** An authenticated user could run arbitrary commands on the host through two Model Context Protocol (MCP) preview endpoints, which accepted `command`, `args`, and `env` values without restriction. The KEV listing followed on June 8, 2026.

Both are fixed in v1.83.7-stable and later, and a [third advisory](https://www.runzero.com/blog/litellm/) chains with them toward remote code execution. Severity ratings under-read in this category, because the component holding every provider key also reads every prompt in plaintext, which means a database read is a credential dump. Who absorbs that patching load is the first thing to settle.

## [Copy link to heading](#every-litellm-alternative-is-a-self-hosted-proxy-or-a-managed-gateway)Every LiteLLM alternative is a self-hosted proxy or a managed gateway

Deployment architecture decides more than product selection does, so it comes first. [Self-hosting](/blog/self-hosted-llm) buys custody. Prompts and provider keys never leave your network, the license cost is zero, and nothing about the request path is opaque to you. The costs land in three places, and they compound as traffic grows rather than showing up on day one:

- **Throughput:** A team running v1.80.15 [reported throughput dropping](https://github.com/BerriAI/litellm/issues/21046) from roughly 16 to 9 requests per second at 500 concurrent requests, and neither disabling spend logs nor adding pgbouncer recovered it. The Python global interpreter lock sets a ceiling that configuration cannot move, which is why [a Rust rewrite](https://github.com/BerriAI/litellm/issues/31263) of the `/chat/completions` path is underway.

- **Latency:** A proxy running in one region adds a full network round-trip in each direction for every request that originates elsewhere. No amount of tuning removes that, and published overhead figures come from the vendors themselves, so they are worth little in a comparison.

- **Operational footprint:** A production deployment is the proxy, PostgreSQL, and [Redis](https://docs.litellm.ai/docs/proxy/redis_requirements), which puts three stateful services on the on-call rotation before any model call happens.

None of that settles the question, because the two models are not exclusive. A self-hosted proxy can front a managed gateway, which keeps the local control plane and hands off provider fan-out:

```
- model_name: gpt-5.4-gateway
  litellm_params:
    model: vercel_ai_gateway/openai/gpt-5.4
    api_key: os.environ/VERCEL_AI_GATEWAY_API_KEY
```

A migration that only has to move provider fan-out is a far smaller project than one replacing the control plane, and the two routinely get scoped as though they were the same job. With that split in hand, the products themselves become comparable.

## [Copy link to heading](#litellm-alternatives-compared-on-license,-hosting,-and-maturity)LiteLLM alternatives compared on license, hosting, and maturity

With the architecture question settled, the field narrows to what each option costs to license, run, and keep:

| Option | License | Self-host | Free tier | Provider coverage | Operational footprint | Maturity signal |
| --- | --- | --- | --- | --- | --- | --- |
| LiteLLM | [MIT license](https://github.com/BerriAI/litellm/blob/main/LICENSE) outside `enterprise/` | Yes | Unlimited when self-hosted | 100+ providers in docs, 140+ on its site | Proxy, PostgreSQL, and Redis | [v1.98.0](https://docs.litellm.ai/release_notes), August 22, 2026 |
| Envoy AI Gateway | [Apache 2.0](https://github.com/envoyproxy/ai-gateway/blob/main/LICENSE) | Yes, Kubernetes only | Unlimited when self-hosted | 16 providers at the v1.0 release | Kubernetes v1.32+, Envoy Gateway, custom resources | [v1.1.0](https://github.com/envoyproxy/ai-gateway/releases), August 21, 2026 |
| Kong AI Gateway | [Apache 2.0 core](https://github.com/Kong/kong/blob/master/LICENSE) | Yes | Open-source tier | Major providers through AI plugins | Kong Gateway cluster | Gateway 3.x generally available, AI Gateway 2.0 in private beta |
| Portkey | [MIT](https://github.com/Portkey-AI/gateway) open-source gateway | Yes | 10,000 logs a month | 45+ providers, 1,600+ models | Docker or npx | Acquired by Palo Alto Networks, May 29, 2026 |
| Helicone | [GPL-3.0](https://github.com/Helicone/ai-gateway) gateway repository | Yes | 10,000 requests a month | 100+ models, vendor-stated | Single binary | Maintenance mode since March 2026 |
| OpenRouter | Proprietary | No | Capped daily requests on free model variants | 70+ providers | None | Live managed service |
| Vercel AI Gateway | Proprietary | No | $5 a month in credits on a model subset | Hundreds of models | None | Available on all plans |

Five of the seven can run inside your own network, and the two that cannot are the only ones with no infrastructure to operate. Those five come first, since they carry the same patching obligation LiteLLM does.

## [Copy link to heading](#self-hosted-alternatives-to-litellm)Self-hosted alternatives to LiteLLM

Kong, Portkey, Helicone, and Envoy AI Gateway all leave the patch cadence with you, exactly as LiteLLM does, so the record that prompted the search applies to them too. Runtime and surrounding infrastructure are where they diverge.

### [Copy link to heading](#envoy-ai-gateway)Envoy AI Gateway

Envoy AI Gateway is a Kubernetes-native gateway with a Go control plane driving Envoy's C++ data plane through custom resource definitions (CRDs). Its first stable release landed in June 2026, which makes it the youngest option here by a wide margin. The Apache 2.0 core ships AWS IAM, Azure Service Principal, and GCP Workload Identity as first-class provider credentials, so cloud-native authentication does not sit behind a commercial tier.

The migration cost is the operating model. Teams already operating [Kubernetes](https://vercel.com/docs/integrations/external-platforms/kubernetes) alongside Envoy or a service mesh extend the infrastructure they patch anyway. Everyone else absorbs a cluster and a CRD-driven configuration model to route model traffic.

### [Copy link to heading](#litellm-vs-envoy-ai-gateway)LiteLLM vs Envoy AI Gateway

LiteLLM and Envoy AI Gateway are compared more than any other pair in this field, and the differences are structural. Configuration is the first split. LiteLLM is a Python application driven by a YAML file, which one engineer can stand up and read. Envoy AI Gateway expresses the same intent as Kubernetes custom resources, which is heavier to start and considerably better at surviving a team change.

Coverage runs the other direction, with 16 providers at Envoy's v1.0 release against LiteLLM's 100+. Identity splits again, since Envoy puts cloud-provider credentials in its open-source core while LiteLLM keeps single sign-on (SSO) and OpenID Connect (OIDC) in [its enterprise tier](https://www.litellm.ai/enterprise).

The operational floor is what usually decides it. LiteLLM needs three stateful services and Envoy AI Gateway needs a current Kubernetes cluster. Whichever of those you already run is the cheaper answer, and neither is worth adopting for the gateway alone.

### [Copy link to heading](#kong-ai-gateway)Kong AI Gateway

Kong layers AI plugins onto the Apache 2.0 Kong Gateway core, where they arrived in Kong Gateway 3.6. The 3.x line is generally available, the AI Gateway 2.0, announced in July 2026, remains in private beta, and the advanced AI plugins and LLM analytics sit in the Enterprise and Konnect tiers.

Where Kong already handles API governance, the plugin model extends a system the team maintains and patches, which is the cleanest adoption path in this group. Kong has not named third-party AI gateway adopters, though, so the evidence base for a production migration is thin. For anyone else, adopting a full API gateway to route model traffic is a large step for a narrow gain.

### [Copy link to heading](#portkey)Portkey

Portkey's open-source gateway is MIT-licensed and carries more routing machinery than anything else here, covering fallbacks and conditional routing, canary testing, circuit-breaker retries, and [50+ guardrails](https://github.com/Portkey-AI/gateway/wiki/Guardrails-on-the-Gateway-Framework). For a team leaving LiteLLM specifically for policy enforcement, this is the closest match in the self-hosted field.

Ownership is the open question. Palo Alto Networks [closed its acquisition](https://www.paloaltonetworks.com/company/press/2026/palo-alto-networks-completes-acquisition-of-portkey-to-secure-ai-agents) of Portkey on May 29, 2026, and Portkey is now the AI Gateway inside Prisma AIRS, which reached general availability in July. The self-hosted v2.0 is labeled Pre-Release, and the open-source repository's last push predates the close.

Both things are true at once. The routing capability is real and available today under MIT. The roadmap for the self-hosted path now runs through an enterprise security platform, and the trillions-of-tokens scale figures the company cites describe the hosted service.

### [Copy link to heading](#helicone)Helicone

Helicone's gateway is a single Rust binary, and it is the lightest thing to operate in this group. The license needs care, because the gateway repository ships under GPL-3.0 even though the broader Helicone platform is Apache 2.0.

Mintlify [acquired Helicone](https://www.mintlify.com/blog/mintlify-acquires-helicone) on March 3, 2026. The stated position is maintenance mode, covering security updates, bug fixes, and new model support with no new feature work, and Mintlify has said it is helping teams migrate to another platform. The gateway repository has not been pushed to since November 2025, and its routing configuration documents fall back without conditional routing or a retry-with-backoff policy.

That makes it a lateral move rather than a migration. Helicone still works well as a stable router with logging attached, which is a reasonable thing to want and a poor thing to build a two-year roadmap on.

## [Copy link to heading](#managed-gateways-as-litellm-alternatives)Managed gateways as LiteLLM alternatives

Vercel AI Gateway and OpenRouter both put a vendor on call for patching and scaling, which resolves the concern that sent most teams searching. What you trade for it is the option to run the gateway on hardware you control.

### [Copy link to heading](#openrouter)OpenRouter

OpenRouter is a proprietary managed service covering 70+ providers, and it has the broadest model catalog in this comparison. Free model variants carry a daily request cap that rises once an account has purchased $10 in lifetime credits.

Every request traverses OpenRouter's layer, which is the deciding factor for anyone with a data residency requirement. The reliability record includes a database outage in August 2025 and two [February 2026 outages](https://openrouter.ai/blog/announcements/openrouter-outages-on-february-17-and-19-2026/) traced to caching dependency failures.

For prototyping with zero operational overhead, OpenRouter fits better than anything else here. Developers who keep it past the prototype stage tend to hold a direct-provider path in reserve for the outage windows.

### [Copy link to heading](#vercel-ai-gateway)Vercel AI Gateway

AI Gateway routes to hundreds of models through a single endpoint at `ai-gateway.vercel.sh/v1`, with no markup on tokens, including when you [bring your own key](https://vercel.com/docs/ai-gateway/authentication-and-byok/byok). The free tier includes $5 a month in credits against a subset of models with lower rate limits, and the paid tier adds the full catalog, BYOK, and custom rate limits.

The routing layer is where the operational difference concentrates. Requests run across 126 points of presence and 20 compute regions rather than one, and automatic cross-provider fallback rescued 3.5% of requests, 5.1% of tokens, and 4.9% of cost through April 2026. Cline A/B tested its previous router against AI Gateway on live traffic for a week. P99 streaming latency [improved 10 to 14%](/blog/cline-on-ai-gateway) and API error rates dropped 43.8%, at under 20ms of added overhead.

[OIDC tokens](https://vercel.com/docs/ai-gateway/authentication-and-byok/oidc) are provisioned automatically for applications deployed on Vercel, so there are no long-lived gateway keys to rotate. [Routing Rules](https://vercel.com/docs/ai-gateway/models-and-providers/routing-rules) rewrite model requests team-wide without a deploy, which covers a provider outage or a model swap without touching application code. [Zero Data Retention](https://vercel.com/docs/ai-gateway/security-and-compliance/zdr) routing is available per request and team-wide. All of it assumes a managed service, so a hard requirement to keep prompts and provider keys on your own infrastructure points back to the five self-hosted options.

## [Copy link to heading](#which-litellm-alternative-fits-your-infrastructure)Which LiteLLM alternative fits your infrastructure

The field sorts cleanly once the choice is framed around infrastructure you already operate, rather than features you might use:

- **Strict data residency, air-gapped, or regulated environments:** Self-hosted LiteLLM, because prompts and keys never cross your network boundary, which is the same reasoning that pushes teams toward [self-hosted models](/blog/self-hosted-llm) in the first place.

- **Existing Kubernetes with Envoy or a service mesh:** Envoy AI Gateway, which extends a control plane the team already runs and patches.

- **Existing Kong API governance:** Kong AI Gateway, through a plugin model the team maintains already.

- **Guardrails and policy enforcement as the primary need:** Portkey, with the roadmap question under new ownership priced in.

- **Broadest model access for rapid prototyping:** OpenRouter, with a direct-provider fallback kept warm.

- **Already deploying on Vercel:** AI Gateway, with no database or cache to operate alongside it, since it runs on the same platform as the [deployments](https://vercel.com/docs/deployments) calling it.

These gateways generally expose OpenAI-compatible APIs, so the mechanical part of a migration is changing a `base_url`. The expensive part is everything that accumulated around the old gateway, including [observability history](/blog/introducing-vercel-drains) that stops at the cutover and routing behavior that encodes decisions nobody wrote down.

## [Copy link to heading](#choose-for-the-team-you-have)Choose for the team you have

Grabbing whichever alternative tops a list is the instinct after an incident. The better question is who patches the gateway next month, because a self-hosted proxy is a security-critical service that holds every provider key and reads every prompt in plaintext. Three exploited vulnerabilities in four months is the cadence that the job now runs at.

A team that can staff that work should self-host, and keep the keys and prompts inside its own network at zero license cost. A team that cannot should hand the patching to a vendor, and accept that the traffic leaves the network to get there. Most teams running LiteLLM today inherited it from whoever prototyped with it, and it earned that position on coverage and configurability that the rest of this field has not matched. The honest answer more often turns on headcount than on features.

For teams choosing the managed path, AI Gateway covers the operational layer:

- **Automatic cross-provider failover:** Fallback routing rescued 3.5% of requests and 5.1% of tokens through April 2026, with no retry logic in application code.

- **OIDC credential injection:** Short-lived tokens replace the static gateway key, so there is no production secret waiting on a rotation schedule.

- **Routing Rules:** Model routing is rewritten team-wide from the dashboard, with no code change and no redeploy.

- **Zero-markup tokens with BYOK:** You pay provider list rates, including on requests that carry your own keys.

- **Zero Data Retention routing:** Requests route through providers under no-retention agreements, per request or team-wide, for workloads that cannot store prompt or completion data.

[Start a new project](https://vercel.com/new) and route your first model call through AI Gateway, or browse [vercel.com/templates](https://vercel.com/templates) for AI applications already wired for production.

## [Copy link to heading](#frequently-asked-questions-about-litellm-alternatives)Frequently asked questions about LiteLLM alternatives

### [Copy link to heading](#which-open-source-ai-gateways-are-production-ready-in-2026)Which open source AI gateways are production-ready in 2026?

LiteLLM at v1.98.0 and Envoy AI Gateway at v1.1.0 are the two with stable releases and active development. Kong Gateway 3.x is generally available, while Kong AI Gateway 2.0 is in private beta, Portkey's self-hosted v2.0 is Pre-Release, and Helicone's gateway is maintenance-only.

### [Copy link to heading](#which-services-provide-a-single-endpoint-for-multiple-ai-model-providers)Which services provide a single endpoint for multiple AI model providers?

Six of the seven options here do: LiteLLM, Portkey, Envoy AI Gateway, Helicone, OpenRouter, and Vercel AI Gateway each expose one OpenAI-compatible endpoint that routes to multiple providers. OpenRouter covers 70+ providers, AI Gateway reaches hundreds of models through `ai-gateway.vercel.sh/v1`, and LiteLLM documents 100+ providers.

### [Copy link to heading](#which-ai-gateways-offer-a-free-tier-or-open-source-version)Which AI gateways offer a free tier or open-source version?

LiteLLM, Envoy AI Gateway, Kong, Portkey, and Helicone are open source and self-hostable, so you pay only for infrastructure and provider tokens. OpenRouter caps daily requests on free model variants, and AI Gateway includes $5 a month in credits at zero markup.

### [Copy link to heading](#what-does-it-cost-to-migrate-off-litellm)What does it cost to migrate off LiteLLM?

Less than teams expect at the API layer and more than they expect everywhere else. Every option here speaks the OpenAI format, so the application code changes by one `base_url`. Spend history, cache keys, and per-team routing rules do not transfer, and rebuilding them is the actual project.