A model gets picked and shipped, and legal review asks what it was trained on. For an open weight model, there is no answer, because the training data was never published.

Downloading the weights is not the same as getting the source code. The weights ship, the recipe that produced them stays behind, and the license can still gate what you build, including what you owe when you publish a fine-tuned checkpoint.

This guide covers what the label grants, which license conditions bind in production, how per-token economics compare against frontier APIs, and what stops working when a closed model gets swapped out.

**Key takeaways:**

- Open weight models publish trained parameters under a license while withholding training code and data, which keeps them outside the Open Source Initiative's definition of open source AI.

- Open weight models ran 36% of AI Gateway token volume in July 2026, up from 11% in April, at roughly a seventh of frontier per-token rates.

- The license that binds you is rarely the family's headline license, since terms change across generations and across model sizes.

- Managed open weight APIs moved the self-hosting break-even past 50 million tokens per day, so most teams are choosing between two managed tiers rather than between building and buying.

- The model string is the only portable part of a migration, because prompt behavior, tool-call parsing, structured output syntax, and content moderation each need separate re-validation.

## [Copy link to heading](#what-are-open-weight-models)What are open weight models?

An open weight model is one you can download and run yourself. The lab publishes the weights, which are the numbers the model learned during training, and holds back the code, data, and process that produced them. You can run those weights on your own hardware, fine-tune them, and serve them to your own users, within whatever the license allows.

That sounds a lot like open source, and the overlap is where teams get into trouble. Software licensing has a settled vocabulary, but it was built for source code, and what gets licensed here is bigger than source code. A team reads "open" and assumes it can audit the training data, or assumes the license permits something it doesn't.

### [Copy link to heading](#open-weight-vs-open-source-comes-down-to-the-training-recipe)Open weight vs open source comes down to the training recipe

Three categories are in circulation. The Open Source AI Definition, or OSAID, from the Open Source Initiative sets the bar for the strictest, and open weight clears only part of it.

On training code, training data, and reproducibility, open weight gives the same answer as closed does:

| Artifact | Closed | Open weight | Open source AI (OSAID) |
| --- | --- | --- | --- |
| Downloadable weights | No, API access only | Yes | Yes |
| License terms | Proprietary terms of service | Custom or permissive, sometimes conditioned on scale or region | OSI-approved |
| Training code | Not released | Not released | Released, including data processing and tokenizers |
| Training data information | Not disclosed | Not disclosed, or partial | Full provenance, scope, filtering, and labeling |
| Reproducible training run | No | No | Yes |
| Representative releases | Frontier lab APIs | Llama 4, DeepSeek V4, gpt-oss, Qwen3.8-27B | OLMo |

Open weight models carry nearly all the production volume, and they are also the ones routinely called open source. Open weights fall short of the four freedoms of use, study, modification, and sharing, which is why the OSI files them separately. The OpenAI gpt-oss releases are unusually careful here and stick to the narrower term. Apache 2.0 covers the weights, and a separate [usage policy](https://help.openai.com/en/articles/11870455-openai-gpt-oss) travels alongside them.

### [Copy link to heading](#the-withheld-recipe-costs-you-provenance-and-reproducibility)The withheld recipe costs you provenance and reproducibility

Without the training code and data, nobody outside the lab can rebuild the model or account for why it behaves the way it does.

Runtime control comes with the weights. Provenance does not:

- **Runtime control:** The weights run wherever you put them, including inside an air-gapped network with no outbound calls. Nothing breaks when a provider has an incident or drops the model from its catalog.

- **Modification rights:** Fine-tuning, quantization, and distillation all work on downloaded weights, and the resulting checkpoint is yours to deploy. A team can shrink a model to fit the hardware it already owns.

- **No data provenance:** The weights say nothing about what went into them, and no amount of probing recovers it. A compliance reviewer asking which personal or copyrighted data the model saw has no way to get an answer.

- **No reproducibility:** The training run cannot be repeated or independently verified. When the model fails on a particular kind of input, you find that out by testing around it, because there is no recipe to inspect.

Those properties decide which of the three categories a team is actually buying into, and the answer usually differs by workload rather than by organization. What changed recently is how fast the open weight category grew.

## [Copy link to heading](#why-the-open-weight-vs-closed-model-split-became-a-routing-decision)Why the open weight vs closed model split became a routing decision

AI Gateway routes tens of trillions of tokens a month, so the Production Index reflects production routing instead of survey answers. In [July 2026](/blog/deepseek-overtakes-google-on-volume-cost-per-token-falls), open weight models ran 36% of gateway token volume, up from 11% in April, at about a seventh of frontier rates. Their share of spending more than doubled to 8.6%, the highest the index has recorded. Anthropic, in the same month, collected 65.1% of all gateway spend on 29.8% of volume, at 4.4 times the average price of every other lab's tokens.

Those two patterns describe different work. Teams route the bulk of their tokens to cheap models and save frontier models for the calls that carry consequences, which is why volume and spend point in opposite directions. Coding agents show that split inside one use case. It is the gateway's largest workload by tokens, and in July, DeepSeek ran nearly a third of the volume there while Anthropic collected more than four of every five dollars spent on it.

The line between those tiers moved in July as well. Z.ai's GLM 5.2 and Moonshot's Kimi K3 became the first open weight models to take a meaningful share of work that closed labs used to own outright. Between them, they account for the entire increase in open weight spend. Among teams that ran more than 10 million tokens in both June and July, three in four changed at least a tenth of their model mix, and three in five changed at least a quarter. Acting on that routing logic starts with knowing which licenses permit it.

## [Copy link to heading](#open-weight-model-licensing-conditions-that-actually-bind)Open weight model licensing conditions that actually bind

Most open weight models generating production volume carry no usage gate at all. The exceptions surface late, usually once a model is already load-bearing, and the name on the license file rarely tells you which kind you have.

A permissive-sounding license name does not guarantee an ungated model:

| Condition type | What triggers it | Typical terms | Where to read it |
| --- | --- | --- | --- |
| No usage gate | Nothing | Commercial use, fine-tuning, and redistribution permitted without thresholds. Apache 2.0 adds a patent grant, MIT does not | [gpt-oss](https://github.com/openai/gpt-oss/blob/main/LICENSE), [Mistral](https://help.mistral.ai/en/articles/347393-under-which-license-are-mistral-s-open-models-available), [Gemma 4](https://ai.google.dev/gemma/docs/core/model_card_4), [DeepSeek V3.2](https://huggingface.co/deepseek-ai/DeepSeek-V3.2/blob/main/LICENSE) |
| Scale gate | Crossing a user-count threshold | Above the threshold, commercial use requires the licensor's approval rather than being permitted by default | [Llama 4](https://github.com/meta-llama/llama-models/blob/main/models/llama4/LICENSE) |
| Revenue gate | Crossing a revenue threshold | Above $20M monthly revenue, Mistral's modified MIT requires a commercial license or use through Mistral Studio | [Devstral 2](https://huggingface.co/mistralai/Devstral-2-123B-Instruct-2512/blob/main/LICENSE) |
| Attached obligations | Distribution or derivative work | Attribution notices, derivative naming rules, region-specific restrictions, or a separate usage policy that travels with the weights | Llama 4 Community License, gpt-oss usage policy |

Apache 2.0 and MIT cover most of the open weight volume on the gateway, so most teams never hit a threshold, and confirming that takes minutes. Attached obligations are the condition that reaches ordinary deployments, since attribution and naming rules apply at any size and follow fine-tuned checkpoints.

Licenses vary within a model family, which is where teams get caught. Gemma 1 through 3 shipped under custom terms with a prohibited-use policy, and Gemma 4 shipped under Apache 2.0 in April 2026. Qwen licenses vary by repository and release rather than following a stable size rule. For example, Qwen3-235B-A22B uses Apache 2.0, while Qwen3.8-Flash-Next uses the Qwen Community License 1.0. Check the license in the exact repository used for deployment. Read the license in the repository you pulled the weights from, not the one in the family's announcement post.

## [Copy link to heading](#open-weight-model-economics-against-frontier-api-pricing)Open weight model economics against frontier API pricing

Licensing tells you what is permitted. Cost tells you what is worth doing.

Prices are per million tokens, input then output:

| Tier | Representative rates | Where it fits |
| --- | --- | --- |
| Frontier closed APIs | [Claude Opus 5](https://claude.com/pricing) at $5 input and $25 output, [GPT-5.6 Sol](https://openai.com/api/pricing) at $4 and $20 | Work where the cost of a wrong answer dwarfs the cost of the tokens |
| Managed open weight APIs | [DeepSeek V4 Flash](https://deepinfra.com/pricing) at $0.08 input and $0.18 output, with other open weight models on managed endpoints in the same band | Variable traffic and high throughput, with no infrastructure to operate |
| Self-hosted | Priced in GPU hours rather than tokens, and highly sensitive to utilization, precision, and serving stack | Data residency and sovereignty requirements, or sustained volume high enough to keep hardware busy |

The GPT-5.6 Sol rate is promotional, guaranteed only through at least November 21, 2026, and the family repriced twice within a month before that. Any budget built on a specific frontier rate needs a re-check date attached to it.

Self-hosting used to be the obvious endgame for open weight adoption, and managed inference changed that arithmetic. Against frontier APIs, reserved GPU economics break even somewhere around two to five million tokens per day. Against managed open weight APIs, the break-even [moves past 50 million](https://www.sitepoint.com/self-hosted-llm-costs-2026) tokens per day. What self-hosting buys is sovereignty and predictable unit cost, paid for with permanent operational headcount and the risk of idle hardware. For most teams, the live question is narrower than build versus buy, and it is which managed tier a given workload belongs in.

## [Copy link to heading](#running-and-switching-open-weight-models-in-production)Running and switching open-weight models in production

Once a workload is assigned to a tier, two questions remain. What serves the model, and what breaks when the model behind an existing feature changes.

### [Copy link to heading](#throughput-numbers-hide-the-serving-stack-tradeoff)Throughput numbers hide the serving-stack tradeoff

Teams running their own GPUs pick an inference engine, and vLLM is the right default for most of them.

Throughput figures come from one 8B benchmark run, so they compare to each other and not to your workload:

| Engine | [Measured throughput](https://www.premai.io/blog/10-best-vllm-alternatives-for-llm-inference-in-production-2026), Llama 3.1 8B | Tradeoff |
| --- | --- | --- |
| [vLLM](https://docs.vllm.ai/en/latest) | 12,553 tokens per second | The production default, with PagedAttention, continuous batching, and 200+ architectures across NVIDIA, AMD, Intel, and tensor processing unit (TPU) hardware |
| SGLang | 16,215 tokens per second | RadixAttention prefix caching drives the advantage, and the advantage shrinks on workloads where prompts rarely repeat |
| TensorRT-LLM | 10,000+ tokens per second | NVIDIA's vendor-specific path, with one to two weeks of setup and single-vendor lock-in |
| Ollama | Not comparable | Local development only, with a default of one parallel request per model |

SGLang's headline number is the one most likely to mislead, because prefix caching pays off on repeated system prompts and returns little on unique-prompt traffic. Benchmark against your own prompt distribution before treating the gap as real. Developers who would rather not operate GPUs at all can serve the same weights per token through a managed endpoint, which removes the engine decision entirely. Teams that want the routing layer itself under their own control have [open source options](https://vercel.com/i/open-source-ai-gateways) as well.

### [Copy link to heading](#behavior-does-not-survive-the-model-swap)Behavior does not survive the model swap

In code, swapping models means editing one string. Everything the model does behind that string has to be re-tested before production traffic moves over:

- **Prompt behavior:** A prompt tuned against one model degrades on another, even when the new model benchmarks higher.

- **Tool-call parsing:** Each model family formats tool calls its own way, and a [parser mismatch](https://fastino.ai/blog/how-to-use-open-weight-models) returns them as plain text with no error raised. On a managed endpoint, the provider owns that configuration, and self-hosting makes it yours.

- **Structured outputs:** Every provider enforces schemas in its own way, through a strict response\_format, a tool-use `input_schema`, or a `response_mime_type` field, and none of it is portable. [AI SDK](https://vercel.com/changelog/ai-sdk-7)'s `generateText` and `streamText` gives one interface across all of them.

- **Content moderation:** Closed providers bundle moderation into the API, and that filtering disappears along with the model. [Llama Guard](https://alphasec.io/content-safety-with-llama-guard-and-groq) is the usual replacement, running as a separate pass alongside whichever model serves the request.

A model that returns fluent text while dropping tool calls still passes a smoke test, so validation needs explicit test cases rather than a spot check.

### [Copy link to heading](#shadow-traffic-settles-it-before-cutover)Shadow traffic settles it before cutover

The question of whether a model works on your traffic has one cheap answer, which is to run it on your traffic without exposing it to users. A slice of production requests goes to the candidate while the primary model keeps serving, and the comparison happens per request rather than in aggregate.

Latency and accuracy come first in that comparison, with cost added second. Leading with cost tends to settle the question before the quality data arrives, and quality is the variable that actually determines whether the cheaper tier is usable for that workload.

## [Copy link to heading](#how-vercel-ai-gateway-helps-teams-run-open-weight-models)How Vercel AI Gateway helps teams run open weight models

Switching between tiers only stays cheap with a routing layer in front of the providers, and the [gateway category](https://vercel.com/i/best-ai-gateways) exists for exactly that. [AI Gateway](https://vercel.com/ai-gateway) provides a single API across hundreds of models, routing to [third-party inference providers](https://vercel.com/docs/ai-gateway/models-and-providers) rather than hosting models itself, with sub-20ms routing latency and no markup on tokens.

### [Copy link to heading](#one-model-string-replaces-one-integration-per-provider)One model string replaces one integration per provider

Every provider a team adds by hand brings its own SDK, authentication scheme, request format, error shapes, and rate-limit behavior. Once that code is spread across the application, changing models stops being a product decision and becomes an infrastructure project, so teams hard-code whichever model they integrated first.

AI Gateway collapses that to a model string. Switching from a frontier model to an open weight one, or between two open weight providers serving the same weights, is a configuration change rather than a new integration. The switch costs the same whether it is a planned tier change or an emergency reroute off a model that stopped being the right pick.

### [Copy link to heading](#failover-covers-a-budget-tier-provider-that-degrades)Failover covers a budget-tier provider that degrades

The cost case for open weight models assumes the cheap tier stays available. A single-provider integration has no path around a degraded endpoint, so a provider incident reaches users as failed generations. Teams end up retreating to a frontier model for reasons that have nothing to do with output quality.

AI Gateway retries the same request on another provider serving the same model, with no code change and no fallback list configured in advance. When [Cline](/blog/cline-on-ai-gateway) A/B tested its previous router against AI Gateway on live production traffic for a week, API error rates dropped 43.8% and streaming latency at the 99th percentile improved by 10 to 14%.

### [Copy link to heading](#cost-attribution-makes-a-shadow-test-conclusive)Cost attribution makes a shadow test conclusive

A homegrown shadow setup usually gets the traffic splitting working and then stalls on attribution, because spend arrives on several provider invoices on different schedules.

Because the gateway sits in the request path, it logs tokens, time to first token, latency, and spend for every request without any instrumentation in your code. The Custom Reporting API groups that by tag, user ID, or quota entity. A shadow test then produces a real per-model cost and latency comparison on the same traffic, which is the evidence a tier decision needs.

### [Copy link to heading](#ai-gateway-ends-where-moderation-begins)AI Gateway ends where moderation begins

Moving requests is not the same as filtering them. Treating a gateway as a safety layer is how teams end up shipping an open weight model with less content filtering than the closed model it replaced. Nothing at the routing layer replaces what the closed provider was bundling.

AI Gateway does not provide built-in guardrails, so content moderation stays with the application team regardless of how requests are routed. Plan that replacement as part of the migration rather than discovering it after cutover, since it is the only one of these that no routing layer can cover for you.

## [Copy link to heading](#route-by-stakes,-not-by-price-list)Route by stakes, not by price list

The model you download today is a snapshot of a catalog that keeps moving. In July, 81% of gateway tokens ran on models that were not on the gateway six months earlier, and the two open weight releases that reached consequential workloads did so within weeks of launch. Against that rate of change, hard-coding a model selection is a liability, because the cost of the decision is not the tokens but the integration work required to undo it. Teams adjusting fastest treat model choice as a routing decision they expect to revisit, which makes the one-string swap the first thing to get right.

Vercel's routing primitives are built so the swap stays cheap enough to repeat:

- **AI Gateway:** One API across hundreds of models, so moving a workload between tiers is a configuration change instead of a new integration.

- **Automatic failover:** A degraded provider triggers a retry on another provider serving the same model, with no fallback list to maintain and no platform fee on tokens.

- **Custom Reporting API:** Cost and latency arrive per request with no tracing layer to build, grouped by tag, user ID, or quota entity, so a shadow test produces real numbers.

- **Per-key spend limits:** A key that exceeds its budget returns an HTTP 402, which stops a runaway agent loop before the spend lands instead of after the invoice arrives.

- **AI SDK** `**generateText**` **and** `**streamText**`**:** One interface over each provider's own way of enforcing a schema, so structured output survives a model swap.

[Start a new project](https://vercel.com/new) to route your first workload through AI Gateway, or [browse Vercel templates](https://vercel.com/templates) for AI applications already wired up for multi-provider routing.

## [Copy link to heading](#frequently-asked-questions-about-open-weight-models)Frequently asked questions about open weight models

### [Copy link to heading](#where-should-i-look-for-open-weight-model-benchmarks)Where should I look for open weight model benchmarks?

Four sources stay current. [LMArena](https://arena.ai/leaderboard/text) measures human preferences across broad tasks, the [Artificial Analysis Intelligence Index](https://artificialanalysis.ai/models) composites nine evaluations, [LiveBench](https://livebench.ai/) refreshes a contamination-free task set, and [SWE-bench Pro](https://www.swebench.com/) covers 1,865 software tasks. Hugging Face's Open LLM Leaderboard has not updated since March 2025, so leave it out of the shortlist.

### [Copy link to heading](#why-do-leaderboards-disagree-on-which-model-is-best)Why do leaderboards disagree on which model is best?

Scores move for reasons unrelated to model quality. [Benchmark contamination](https://arxiv.org/html/2506.21614v1) inflates results when web-scraped training data includes benchmark samples, [arena data](https://neurips.cc/virtual/2025/loc/san-diego/poster/121845) is unevenly distributed across labs, and rank order on MMLU (Massive Multitask Language Understanding) [shifts with prompt template](https://arxiv.org/html/2406.01574v2) and scoring function. Use leaderboards to build a shortlist, then evaluate on your own workload.

### [Copy link to heading](#can-you-fine-tune-open-weight-models)Can you fine-tune open weight models?

Yes, with conditions that vary by family. Apache 2.0 and MIT models permit fine-tuning without usage gates. Licenses with derivative-naming or attribution clauses, including the Llama 4 Community License, apply to published checkpoints. Hugging Face PEFT (parameter-efficient fine-tuning) and [LlamaFactory](https://github.com/hiyouga/LlamaFactory) cover LoRA (low-rank adaptation) and its quantized variant across 100+ architectures.

### [Copy link to heading](#are-open-weight-models-production-ready)Are open-weight models production-ready?

For high-throughput work, they already are. Roughly one in eight enterprise teams on AI Gateway runs an open weight model in production, and open weight token share passed a third of gateway volume in July 2026. We run [a fine-tuned model](/blog/v0-composite-model-family) in production ourselves, and frontier models still hold the work where errors are expensive.