Reasoning models and standard LLMs answer the same request in different ways, and the difference is where the compute goes. A standard model resolves a request in one pass per output token. A reasoning model spends extra compute before it answers, which makes its cost and latency depend on the difficulty of the input rather than the volume of traffic. This guide covers what separates the two and how to decide which one handles a given request.

## [Copy link to heading](#key-takeaways)Key takeaways

- Reasoning models are post-trained to generate intermediate steps before answering, and they can revise those steps instead of committing to the first path they take.

- A standard model bills against the answer you see. A reasoning model bills against hidden work that happens first, so its cost tracks input difficulty rather than request volume.

- Extended reasoning pays off where a task decomposes into checkable steps. On classification, extraction, and summarization, it buys almost nothing.

- One routing study held 95% of GPT-4's MT Bench score while sending only 26% of calls to it, which is the shape of the argument for treating extra compute as an escalation path rather than a default.

- Three things determine whether a reasoning route is operable: the execution window, the effort cap, and per-request token telemetry. Each needs its own policy, separate from the rest of the application.

## [Copy link to heading](#what-are-reasoning-models)What are reasoning models?

A reasoning model, also called a reasoning language model (RLM), a large reasoning model (LRM), or informally a thinking large language model (LLM), is an LLM post-trained to generate intermediate reasoning steps before producing a final answer. Standard models produce a response immediately. Reasoning models allocate additional compute, often called thinking time, to problems that take more than one step to solve.

OpenAI introduced the terminology in September 2024 with the [o1 series](https://arxiv.org/abs/2412.16720), describing the models as designed to spend more time thinking before responding. Two properties define the class. These models can revisit and revise earlier reasoning steps rather than following a single forward path, and they treat computation at inference time as a way to scale performance, complementing the older scaling levers of training data, parameter count, and training compute.

### [Copy link to heading](#what-are-the-differences-between-reasoning-models-and-standard-llms)What are the differences between reasoning models and standard LLMs?

The difference that matters operationally is where the budget comes from. A standard model's cost and latency track the response the user sees. A reasoning model decides how long to think per request, so the route's duration is set by what arrives rather than by how it is configured.

Here are the six dimensions that separate the two classes in production:

| Dimension | Standard LLMs | Reasoning models |
| --- | --- | --- |
| Where compute is spent | One forward pass per output token | Thinking tokens generated before the final answer |
| Latency profile | Short interactive completions in a predictable band | Longer completions that vary with difficulty and effort setting |
| Cost driver | Visible output token count | Hidden reasoning tokens billed at the output rate |
| Output shape | Direct answer, a better fit for structured output | Thinking may stay hidden or arrive summarized, so format control takes more work |
| Failure mode | Skips logical steps while sounding confident | Overthinks single-step tasks and can override stated constraints |
| Task fit | Classification, extraction, summarization, real-time chat | Multi-step math and code, multi-hop retrieval, agentic planning |

Every row turns a generic model choice into a per-request purchase decision. A direct extraction request has almost no sequential work to serialize, while a proof or a bug diagnosis can repay the extra steps.

### [Copy link to heading](#why-reasoning-models-matter-for-teams-shipping-llm-features)Why reasoning models matter for teams shipping LLM features

Adding a reasoning model changes the operating profile of a route, not just the quality of its answers. Four consequences are worth planning for before the first reasoning call ships:

- **One feature now needs two models:** Most requests go to a standard model and the hard ones escalate, so the choice moves into a router instead of a constant at the top of the file.

- **A monthly request count stops predicting the invoice:** Two requests with identical visible output can burn very different amounts of hidden generation, so spend follows difficulty, not volume.

- **The interface has to handle a blank pause:** Time to first token stretches while the model thinks, so the UI needs a plan for the stretch before any output appears.

- **The route needs something to check against:** Extended reasoning only pays off where a step is verifiable, which usually means a test, a schema, or a retrieval result sits in the loop.

None of this argues against reasoning models. It argues for giving a reasoning route its own configuration instead of the application's defaults.

## [Copy link to heading](#how-reasoning-models-work)How reasoning models work

Reasoning models spend their extra compute generating intermediate tokens before the final answer, and each of those tokens is another full forward pass through the network. Five properties of that mechanism decide which tasks repay the cost.

### [Copy link to heading](#1.-test-time-compute-as-a-second-scaling-dimension)1\. Test-time compute as a second scaling dimension

A standard model's capability is fixed once training ends, because each output token comes from a single forward pass through the network. Reasoning models add a lever that operates after training, at the moment of the request.

Each thinking token is another full forward pass. Generating a hundred thinking tokens before the answer means a hundred additional passes, which is why the approach is called test-time compute or inference-time compute. It scales performance without retraining, and it bills per request rather than per model.

Training compute is a cost the provider paid once, before the model shipped. Test-time compute is a cost you pay on every request, which is why it belongs in a budget rather than a capacity plan.

### [Copy link to heading](#2.-chain-of-thought-serializes-work-one-forward-pass-cannot-do)2\. Chain-of-thought serializes work one forward pass cannot do

Wei et al. formalized [chain-of-thought prompting](https://arxiv.org/abs/2201.11903) in 2022 as intermediate reasoning tokens generated between the question and the answer. The mechanism is sequential rather than magical. A single forward pass computes a fixed amount, and writing intermediate steps into the context lets the next pass build on the last one.

Gains concentrate on problems with sequential structure for that reason. A task that decomposes into ordered steps, where each step depends on the one before it, gets more out of a longer chain. A task that resolves in one step gets a longer response and the same answer.

This is the mechanism behind most disappointing reasoning deployments. Teams enable high effort on a route that has no sequential work to serialize, then conclude the model class underdelivers.

### [Copy link to heading](#3.-backtracking-separates-the-class-from-prompted-step-by-step-output)3\. Backtracking separates the class from prompted step-by-step output

Ask a standard model to think step by step and it walks in a straight line, one step after the next. A reasoning model can stop halfway, back up, and try a different route. That ability to revise, not the visible list of steps, is what actually separates the two.

The capability is real and narrower than it sounds. Treat backtracking as a reason these models handle multi-step problems better, not as a guarantee that a long chain will recover from a wrong turn. A route that depends on recovery still needs a check outside the model.

### [Copy link to heading](#4.-effort-levels-turn-the-thinking-budget-into-a-routing-parameter)4\. Effort levels turn the thinking budget into a routing parameter

The thinking budget is exposed as a control rather than a fixed model property, which is what makes reasoning tractable to route. Through [AI Gateway's reasoning support](https://vercel.com/docs/ai-gateway/models-and-providers/reasoning), the `effort` parameter accepts `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`, and the gateway maps that value to each provider's native format, whether that is a reasoning effort setting, a thinking budget, or a thinking configuration block.

Two mechanics matter when wiring this up. The `effort` and `max_tokens` parameters are mutually exclusive, so a route picks one control rather than both. Newer [Anthropic models use adaptive thinking](https://vercel.com/docs/ai-gateway/models-and-providers/reasoning/anthropic), where the model decides how much to think based on an effort level, and Claude Opus 4.7 and later require it because the older fixed-budget API is no longer accepted.

Once thinking depth is configuration rather than a fixed model property, the decision moves to which requests deserve it.

## [Copy link to heading](#when-to-use-reasoning-models-instead-of-standard-llms)When to use reasoning models instead of standard LLMs

Route to a reasoning model when the task breaks into steps something can check, and to a standard model when it doesn't. That single test resolves most requests. Evaluations of [RouteLLM](https://arxiv.org/abs/2406.18665) point the same way: a trained router retained 95% of GPT-4's score on MT Bench while sending 26% of calls to it, with the rest going to a weaker model. That study paired a strong and a weak model rather than a reasoning and a standard one, so it isn't a reasoning benchmark, but the shape holds. Most traffic doesn't need the expensive path.

For choosing among specific models once the tier is settled, see the guide to the [best AI models for developers](https://vercel.com/i/best-ai-models-for-developers).

### [Copy link to heading](#when-the-task-breaks-into-checkable-steps)When the task breaks into checkable steps

Reasoning models repay their cost when a task decomposes into ordered steps and something outside the model can verify progress. Verifiability is the stronger of the two signals, because it turns extra tokens into checked progress rather than length.

#### [Copy link to heading](#multi-step-math-and-proofs)Multi-step math and proofs

Each step follows from the last and can be checked against it, so a longer chain accumulates verified progress.

#### [Copy link to heading](#code-debugging-against-a-test-suite)Code debugging against a test suite

A failing test gives the model a target and a signal. Extended reasoning converges rather than wandering, because each hypothesis produces a result the model can read before proposing the next one.

#### [Copy link to heading](#multi-hop-retrieval)Multi-hop retrieval

Connecting evidence across several documents is sequential work by nature. The model has to hold an intermediate conclusion, retrieve against it, then revise, which is what thinking tokens serialize.

#### [Copy link to heading](#agentic-planning)Agentic planning

Decomposing a goal into ordered steps benefits from reasoning the model can revise before committing to a tool call. The cost of a bad plan is several wasted calls rather than one wrong sentence.

### [Copy link to heading](#when-a-wrong-answer-costs-more-than-the-wait)When a wrong answer costs more than the wait

Some routes justify a reasoning model on consequence rather than on task structure. A schema migration, a production incident summary, or a financial calculation carries an error cost high enough that a longer response window is worth paying for.

The tradeoff is direct. Escalating buys a better answer and costs a longer wait plus a larger token budget. A high-stakes route still needs a maximum effort level, because consequence justifies more thinking and does not justify unbounded thinking.

### [Copy link to heading](#when-escalating-backfires)When escalating backfires

Every reasoning route inherits a set of failure modes that standard completions do not have. Naming them is what makes the route operable, because each one has a different mitigation.

The limitations that shape how a reasoning route gets deployed:

- **Overthinking on single-step tasks:** Hundreds of thinking tokens spent on a question a standard model answers in one pass, with no accuracy gain to show for the spend.

- **Reasoning rigidity:** Research on [instruction overriding](https://arxiv.org/abs/2505.17225) documents models defaulting to a familiar solution template on subtly modified versions of well-known problems, overriding conditions the prompt states explicitly. This differs from hallucination, since nothing is invented, and from prompt brittleness, since the instruction is read correctly and then discarded in favor of an ingrained pattern.

- **Unfaithful reasoning traces:** Work on [reasoning faithfulness](https://arxiv.org/abs/2505.05410) found that the models available at the time of that study acknowledged embedded hints in a minority of cases, which makes the trace a debugging aid rather than an audit record.

- **Cost that tracks difficulty rather than volume:** Hidden reasoning tokens bill at the output rate, so spend becomes a function of what users send rather than how often they send it.

- **Latency that varies per request:** Time to first token stretches with the thinking phase instead of holding a band, which changes what the interface has to do while waiting.

Early errors compound on top of all of this, because a wrong assumption in the first few steps carries through the rest of the chain and arrives wearing the confidence of a long derivation. These constraints are what turn model selection into a routing problem, and routing is only half of what a production reasoning route needs.

## [Copy link to heading](#where-standard-llms-remain-the-better-default)Where standard LLMs remain the better default

Most production traffic belongs on a standard model, and framing that as a compromise gets the economics backward. A standard model resolves a request through one forward pass per output token, which is why its cost and latency track the visible response rather than the difficulty of the input.

### [Copy link to heading](#the-four-routes-that-belong-on-a-standard-model)The four routes that belong on a standard model

Four kinds of work resolve faster and cheaper on the direct path, and extended reasoning on any of them adds latency and spend without adding accuracy:

- **Classification:** A fixed label set gives the model a bounded decision, and there is no sequential work for thinking tokens to serialize.

- **Extraction into a fixed schema:** The answer is already present in the input, so the task is locating and formatting it rather than deriving it.

- **Summarization:** Compression resolves in a single pass when the source material is already in the context window.

- **Real-time chat:** Time to first token governs how the product feels, and a thinking phase inserts a blank pause before any visible output.

### [Copy link to heading](#why-format-strict-routes-are-the-clearest-case)Why format-strict routes are the clearest case

Standard models hold a schema more reliably. Reasoning content that reaches the final answer can break a parser downstream, so strict JSON output belongs on the direct path unless evaluation data says otherwise. A route that needs both extended reasoning and guaranteed structure needs a validation step between the model and the consumer, which is additional work a direct route does not require.

### [Copy link to heading](#the-one-case-where-the-default-is-wrong)The one case where the default is wrong

A standard model answers directly, which means it can skip a logical step and still sound confident, and nothing in its output signals that a step was skipped. On decomposable problems where each step depends on the last, that is exactly the failure a reasoning model's extra passes exist to prevent. It is the one condition that overrides everything above.

Everywhere else, keep the route on a standard model until measured results say otherwise, and treat escalation as the exception that has to earn its place.

## [Copy link to heading](#four-practices-for-running-a-reasoning-model-route-in-production)Four practices for running a reasoning model route in production

Routing decides which requests get extra compute. These four decide whether the resulting route stays operable, and all of them hold regardless of which platform the route runs on.

### [Copy link to heading](#1.-cap-effort-before-the-request-reaches-the-provider)1\. Cap effort before the request reaches the provider

Without a ceiling, thinking depth is set by whatever arrives in the prompt, and a single unusual input can consume a multiple of the tokens a typical request uses. Set effort low by default and raise it per route on measured gains, so an escalation goes to a defined level rather than to whatever the model decides it needs.

### [Copy link to heading](#2.-record-hidden-reasoning-tokens-per-request)2\. Record hidden reasoning tokens per request

Logs that capture only response text miss most of what a reasoning route costs. Providers report reasoning token counts separately, and attributing those counts to routes rather than to the application is what surfaces a cost anomaly early. A route that spends hidden tokens without improving evaluated answers is the signal to lower its effort or send that traffic back to a standard model.

### [Copy link to heading](#3.-track-time-to-first-token-separately-from-total-duration)3\. Track time to first token separately from total duration

A reasoning request can look healthy on total duration while the interface shows nothing, because the thinking phase produces no visible output. Aggregate latency dashboards hide that gap. Measuring time to first token per model and per route is what tells a team whether a delay came from reasoning effort or from downstream work, and it is the metric a user actually experiences.

### [Copy link to heading](#4.-move-the-longest-calls-off-the-request-thread)4\. Move the longest calls off the request thread

A high-effort call can outlast the execution window a route was configured for, and the timeout arrives after most of the model work is already done and paid for. Treat the longest reasoning work as a job rather than a request: stream from the first visible token, and queue anything that cannot finish inside the window.

The rest of what a reasoning route needs, long-running execution, spend enforcement, and provider failover, depends on the platform underneath it.

## [Copy link to heading](#how-vercel-powers-reasoning-model-routing-for-engineering-teams)How Vercel powers reasoning model routing for engineering teams

Vercel puts model routing next to long-running execution and the observability that makes routing decisions measurable. A routing policy depends on all three: long calls need an execution window sized for them, per-request token data has to be available without custom instrumentation, and a degraded provider cannot be allowed to take the route down.

### [Copy link to heading](#ai-gateway-routes-requests-between-tiers)AI Gateway routes requests between tiers

Teams without a dedicated ML infrastructure team tend to converge on one model for everything, because wiring per-request model selection into feature code never reaches the top of a sprint. The result is frontier pricing on classification traffic.

[AI Gateway](https://vercel.com/docs/ai-gateway) provides a single endpoint across providers, so the model a request uses becomes a string in runtime configuration rather than a provider integration in feature code. The [cost-aware routing pattern](https://vercel.com/kb/guide/cost-aware-model-routing-with-ai-gateway) documents the shape this takes in practice, classifying request difficulty, resolving a tier to a model string, and escalating only when a cheaper answer fails a verifiable check.

Gateway budgets are set per API key, and a key that exceeds its limit returns a 402 instead of passing the request through. Give each tier its own key, and a misrouting classifier produces rejected requests rather than a surprise invoice.

### [Copy link to heading](#fluid-compute-runs-long-reasoning-calls)Fluid compute runs long reasoning calls

Intermittent timeouts on high-effort routes usually mean the execution window was inherited from an application default rather than chosen for what the route does. The thinking phase does not fit inside a window sized for a database query.

With [Fluid compute](https://vercel.com/docs/fluid-compute), [Vercel Functions](https://vercel.com/docs/functions/configuring-functions/duration) default to 300 seconds on every plan and run up to 800 seconds on Pro and Enterprise. An extended maximum of 30 minutes is in beta for supported Node.js and Python runtime versions, set through function-level configuration. Because a reasoning call spends most of its window waiting on the provider rather than running code, the waiting time does not bill as active compute.

Work that needs no ceiling belongs in [Vercel Workflows](https://vercel.com/docs/workflows), which holds state across pauses without a duration limit.

### [Copy link to heading](#gateway-observability-shows-what-made-a-request-slow)Gateway observability shows what made a request slow

A slow request has two plausible explanations, model time or tool time, and telling them apart takes instrumentation that separates the two. Most teams build that view only after the first incident where nobody could say which one it was.

[AI Gateway reports time to first token and token counts by model](https://vercel.com/docs/ai-gateway/observability-and-spend) without additional instrumentation, available at both team and project level, which covers the reasoning-specific metrics that aggregate application dashboards hide. The Custom Reporting API breaks the same data down by model, user, tag, provider, or credential type when finer attribution is the question.

[Vercel Drains](https://vercel.com/docs/drains) streams logs and exports traces in OpenTelemetry format. Teams already running an observability backend get reasoning telemetry in the same place as their application and infrastructure traces, instead of in a separate tool.

### [Copy link to heading](#model-fallbacks-keep-a-route-serving)Model fallbacks keep a route serving

A reasoning request carries generated tokens and often tool state, so an error partway through costs more than a failed completion, and handling it in application code means provider-specific error branches piling up around the generation logic.

AI Gateway falls back to alternate providers automatically when the primary is unavailable, and a [model fallbacks](https://vercel.com/docs/ai-gateway/models-and-providers/model-fallbacks) array covers the case where a model itself is unavailable. Provider ordering lives in application code, version-controlled alongside the generation logic:

typescript

```
import { streamText } from 'ai';

const result = streamText({
  model: 'anthropic/claude-opus-5',
  prompt: 'Explain the tradeoffs of server-side rendering',
  providerOptions: {
    gateway: {
      order: ['vertex', 'bedrock', 'anthropic'],
    },
  },
});

for await (const chunk of result.textStream) {
  process.stdout.write(chunk);
}
```

Ordering providers this way stops a degraded primary from taking the whole route down. Test that path on purpose, though, because an untested fallback is a config file, not a guarantee.

## [Copy link to heading](#route-reasoning-models-by-task-on-vercel)Route reasoning models by task on Vercel

The support router that pays reasoning prices on classification traffic and gains no accuracy has a routing problem, not a model problem: it runs a budget that scales with difficulty as though it scaled with traffic. Applying the reasoning-versus-standard choice per request is what separates a bill that tracks usage from one that tracks whatever users happened to send. Classify the task first, escalate on a defined signal, and measure the escalation's cost separately from the rest of the application.

Vercel provides the primitives that a routing policy depends on:

- **AI Gateway:** One endpoint across providers, so escalation policy lives in runtime configuration rather than in feature code, with per-key budgets that reject over-limit requests instead of billing them.

- **Model fallbacks:** An ordered provider list plus automatic failover, so a reasoning route keeps serving when the primary provider degrades.

- **Fluid compute:** Execution windows sized for long reasoning calls, where provider wait time does not bill as active compute.

- **Extended function duration:** 300 seconds by default on every plan, 800 seconds on Pro and Enterprise, and a 30-minute extended maximum in beta for long reasoning and tool calls.

- **Gateway observability and Vercel Drains:** Time to first token and token counts by model at the team and project level, with OpenTelemetry export into the backend that the team already runs.

Deploy a routed AI feature by [starting a new project](https://vercel.com/new), or browse [Vercel templates](https://vercel.com/templates) for a working starting point.

## [Copy link to heading](#frequently-asked-questions-about-reasoning-models)Frequently asked questions about reasoning models

### [Copy link to heading](#do-reasoning-models-always-outperform-standard-llms)Do reasoning models always outperform standard LLMs?

No. Gains concentrate on multi-step math, code, multi-hop retrieval, and agentic planning. Standard models match them at lower cost on classification, extraction, and summarization, where extended reasoning adds latency and spend without adding accuracy.

### [Copy link to heading](#what-is-the-difference-between-chain-of-thought-prompting-and-a-reasoning-model)What is the difference between chain-of-thought prompting and a reasoning model?

Chain-of-thought prompting asks a standard model to show intermediate steps at inference time. A reasoning model is post-trained through fine-tuning, reinforcement learning, or distillation to generate and revise those steps by default, including the ability to abandon a path and try another.

### [Copy link to heading](#why-is-a-reasoning-model's-bill-higher-than-its-visible-output-suggests)Why is a reasoning model's bill higher than its visible output suggests?

Hidden reasoning tokens bill at the output rate even when they never appear in the response. Anthropic's Claude 4 models, for example, return summarized thinking output while billing the full thinking tokens, so visible length is a poor proxy for what a request costs.

### [Copy link to heading](#when-should-a-request-go-to-a-standard-model-instead)When should a request go to a standard model instead?

Route to a standard model when the task is single-step, latency-bound, or format-strict. Classification, extraction, summarization, and real-time chat are resolved through a shorter generation path, and standard models hold output schemas more reliably than reasoning models do.

### [Copy link to heading](#how-long-should-a-reasoning-call-be-allowed-to-run)How long should a reasoning call be allowed to run?

Long enough for the effort level the route sets, and no longer. On Vercel, functions default to 300 seconds with a maximum of 800 seconds on Pro and Enterprise, plus a 30-minute extended maximum in beta. Unbounded work belongs in Vercel Workflows.