Reasoning models and standard LLMs answer the same request in different ways, and the difference is where the compute goes. A standard model resolves a request in one pass per output token. A reasoning model spends extra compute before it answers, which makes its cost and latency depend on the difficulty of the input rather than the volume of traffic. This guide covers what separates the two and how to decide which one handles a given request.
Copy link to headingKey takeaways
Reasoning models are post-trained to generate intermediate steps before answering, and they can revise those steps instead of committing to the first path they take.
A standard model bills against the answer you see. A reasoning model bills against hidden work that happens first, so its cost tracks input difficulty rather than request volume.
Extended reasoning pays off where a task decomposes into checkable steps. On classification, extraction, and summarization, it buys almost nothing.
One routing study held 95% of GPT-4's MT Bench score while sending only 26% of calls to it, which is the shape of the argument for treating extra compute as an escalation path rather than a default.
Three things determine whether a reasoning route is operable: the execution window, the effort cap, and per-request token telemetry. Each needs its own policy, separate from the rest of the application.
Copy link to headingWhat are reasoning models?
A reasoning model, also called a reasoning language model (RLM), a large reasoning model (LRM), or informally a thinking large language model (LLM), is an LLM post-trained to generate intermediate reasoning steps before producing a final answer. Standard models produce a response immediately. Reasoning models allocate additional compute, often called thinking time, to problems that take more than one step to solve.
OpenAI introduced the terminology in September 2024 with the o1 series, describing the models as designed to spend more time thinking before responding. Two properties define the class. These models can revisit and revise earlier reasoning steps rather than following a single forward path, and they treat computation at inference time as a way to scale performance, complementing the older scaling levers of training data, parameter count, and training compute.
Copy link to headingWhat are the differences between reasoning models and standard LLMs?
The difference that matters operationally is where the budget comes from. A standard model's cost and latency track the response the user sees. A reasoning model decides how long to think per request, so the route's duration is set by what arrives rather than by how it is configured.
Here are the six dimensions that separate the two classes in production:
Every row turns a generic model choice into a per-request purchase decision. A direct extraction request has almost no sequential work to serialize, while a proof or a bug diagnosis can repay the extra steps.
Copy link to headingWhy reasoning models matter for teams shipping LLM features
Adding a reasoning model changes the operating profile of a route, not just the quality of its answers. Four consequences are worth planning for before the first reasoning call ships:
One feature now needs two models: Most requests go to a standard model and the hard ones escalate, so the choice moves into a router instead of a constant at the top of the file.
A monthly request count stops predicting the invoice: Two requests with identical visible output can burn very different amounts of hidden generation, so spend follows difficulty, not volume.
The interface has to handle a blank pause: Time to first token stretches while the model thinks, so the UI needs a plan for the stretch before any output appears.
The route needs something to check against: Extended reasoning only pays off where a step is verifiable, which usually means a test, a schema, or a retrieval result sits in the loop.
None of this argues against reasoning models. It argues for giving a reasoning route its own configuration instead of the application's defaults.
Copy link to headingHow reasoning models work
Reasoning models spend their extra compute generating intermediate tokens before the final answer, and each of those tokens is another full forward pass through the network. Five properties of that mechanism decide which tasks repay the cost.
Copy link to heading1. Test-time compute as a second scaling dimension
A standard model's capability is fixed once training ends, because each output token comes from a single forward pass through the network. Reasoning models add a lever that operates after training, at the moment of the request.
Each thinking token is another full forward pass. Generating a hundred thinking tokens before the answer means a hundred additional passes, which is why the approach is called test-time compute or inference-time compute. It scales performance without retraining, and it bills per request rather than per model.
Training compute is a cost the provider paid once, before the model shipped. Test-time compute is a cost you pay on every request, which is why it belongs in a budget rather than a capacity plan.
Copy link to heading2. Chain-of-thought serializes work one forward pass cannot do
Wei et al. formalized chain-of-thought prompting in 2022 as intermediate reasoning tokens generated between the question and the answer. The mechanism is sequential rather than magical. A single forward pass computes a fixed amount, and writing intermediate steps into the context lets the next pass build on the last one.
Gains concentrate on problems with sequential structure for that reason. A task that decomposes into ordered steps, where each step depends on the one before it, gets more out of a longer chain. A task that resolves in one step gets a longer response and the same answer.
This is the mechanism behind most disappointing reasoning deployments. Teams enable high effort on a route that has no sequential work to serialize, then conclude the model class underdelivers.
Copy link to heading3. Backtracking separates the class from prompted step-by-step output
Ask a standard model to think step by step and it walks in a straight line, one step after the next. A reasoning model can stop halfway, back up, and try a different route. That ability to revise, not the visible list of steps, is what actually separates the two.
The capability is real and narrower than it sounds. Treat backtracking as a reason these models handle multi-step problems better, not as a guarantee that a long chain will recover from a wrong turn. A route that depends on recovery still needs a check outside the model.
Copy link to heading4. Effort levels turn the thinking budget into a routing parameter
The thinking budget is exposed as a control rather than a fixed model property, which is what makes reasoning tractable to route. Through AI Gateway's reasoning support, the effort parameter accepts none, minimal, low, medium, high, and xhigh, and the gateway maps that value to each provider's native format, whether that is a reasoning effort setting, a thinking budget, or a thinking configuration block.
Two mechanics matter when wiring this up. The effort and max_tokens parameters are mutually exclusive, so a route picks one control rather than both. Newer Anthropic models use adaptive thinking, where the model decides how much to think based on an effort level, and Claude Opus 4.7 and later require it because the older fixed-budget API is no longer accepted.
Once thinking depth is configuration rather than a fixed model property, the decision moves to which requests deserve it.
Copy link to headingWhen to use reasoning models instead of standard LLMs
Route to a reasoning model when the task breaks into steps something can check, and to a standard model when it doesn't. That single test resolves most requests. Evaluations of RouteLLM point the same way: a trained router retained 95% of GPT-4's score on MT Bench while sending 26% of calls to it, with the rest going to a weaker model. That study paired a strong and a weak model rather than a reasoning and a standard one, so it isn't a reasoning benchmark, but the shape holds. Most traffic doesn't need the expensive path.
For choosing among specific models once the tier is settled, see the guide to the best AI models for developers.
Copy link to headingWhen the task breaks into checkable steps
Reasoning models repay their cost when a task decomposes into ordered steps and something outside the model can verify progress. Verifiability is the stronger of the two signals, because it turns extra tokens into checked progress rather than length.
Copy link to headingMulti-step math and proofs
Each step follows from the last and can be checked against it, so a longer chain accumulates verified progress.
Copy link to headingCode debugging against a test suite
A failing test gives the model a target and a signal. Extended reasoning converges rather than wandering, because each hypothesis produces a result the model can read before proposing the next one.
Copy link to headingMulti-hop retrieval
Connecting evidence across several documents is sequential work by nature. The model has to hold an intermediate conclusion, retrieve against it, then revise, which is what thinking tokens serialize.
Copy link to headingAgentic planning
Decomposing a goal into ordered steps benefits from reasoning the model can revise before committing to a tool call. The cost of a bad plan is several wasted calls rather than one wrong sentence.
Copy link to headingWhen a wrong answer costs more than the wait
Some routes justify a reasoning model on consequence rather than on task structure. A schema migration, a production incident summary, or a financial calculation carries an error cost high enough that a longer response window is worth paying for.
The tradeoff is direct. Escalating buys a better answer and costs a longer wait plus a larger token budget. A high-stakes route still needs a maximum effort level, because consequence justifies more thinking and does not justify unbounded thinking.
Copy link to headingWhen escalating backfires
Every reasoning route inherits a set of failure modes that standard completions do not have. Naming them is what makes the route operable, because each one has a different mitigation.
The limitations that shape how a reasoning route gets deployed:
Overthinking on single-step tasks: Hundreds of thinking tokens spent on a question a standard model answers in one pass, with no accuracy gain to show for the spend.
Reasoning rigidity: Research on instruction overriding documents models defaulting to a familiar solution template on subtly modified versions of well-known problems, overriding conditions the prompt states explicitly. This differs from hallucination, since nothing is invented, and from prompt brittleness, since the instruction is read correctly and then discarded in favor of an ingrained pattern.
Unfaithful reasoning traces: Work on reasoning faithfulness found that the models available at the time of that study acknowledged embedded hints in a minority of cases, which makes the trace a debugging aid rather than an audit record.
Cost that tracks difficulty rather than volume: Hidden reasoning tokens bill at the output rate, so spend becomes a function of what users send rather than how often they send it.
Latency that varies per request: Time to first token stretches with the thinking phase instead of holding a band, which changes what the interface has to do while waiting.
Early errors compound on top of all of this, because a wrong assumption in the first few steps carries through the rest of the chain and arrives wearing the confidence of a long derivation. These constraints are what turn model selection into a routing problem, and routing is only half of what a production reasoning route needs.
Copy link to headingWhere standard LLMs remain the better default
Most production traffic belongs on a standard model, and framing that as a compromise gets the economics backward. A standard model resolves a request through one forward pass per output token, which is why its cost and latency track the visible response rather than the difficulty of the input.
Copy link to headingThe four routes that belong on a standard model
Four kinds of work resolve faster and cheaper on the direct path, and extended reasoning on any of them adds latency and spend without adding accuracy:
Classification: A fixed label set gives the model a bounded decision, and there is no sequential work for thinking tokens to serialize.
Extraction into a fixed schema: The answer is already present in the input, so the task is locating and formatting it rather than deriving it.
Summarization: Compression resolves in a single pass when the source material is already in the context window.
Real-time chat: Time to first token governs how the product feels, and a thinking phase inserts a blank pause before any visible output.
Copy link to headingWhy format-strict routes are the clearest case
Standard models hold a schema more reliably. Reasoning content that reaches the final answer can break a parser downstream, so strict JSON output belongs on the direct path unless evaluation data says otherwise. A route that needs both extended reasoning and guaranteed structure needs a validation step between the model and the consumer, which is additional work a direct route does not require.
Copy link to headingThe one case where the default is wrong
A standard model answers directly, which means it can skip a logical step and still sound confident, and nothing in its output signals that a step was skipped. On decomposable problems where each step depends on the last, that is exactly the failure a reasoning model's extra passes exist to prevent. It is the one condition that overrides everything above.
Everywhere else, keep the route on a standard model until measured results say otherwise, and treat escalation as the exception that has to earn its place.
Copy link to headingFour practices for running a reasoning model route in production
Routing decides which requests get extra compute. These four decide whether the resulting route stays operable, and all of them hold regardless of which platform the route runs on.
Copy link to heading1. Cap effort before the request reaches the provider
Without a ceiling, thinking depth is set by whatever arrives in the prompt, and a single unusual input can consume a multiple of the tokens a typical request uses. Set effort low by default and raise it per route on measured gains, so an escalation goes to a defined level rather than to whatever the model decides it needs.
2. Record hidden reasoning tokens per request
Logs that capture only response text miss most of what a reasoning route costs. Providers report reasoning token counts separately, and attributing those counts to routes rather than to the application is what surfaces a cost anomaly early. A route that spends hidden tokens without improving evaluated answers is the signal to lower its effort or send that traffic back to a standard model.
Copy link to heading3. Track time to first token separately from total duration
A reasoning request can look healthy on total duration while the interface shows nothing, because the thinking phase produces no visible output. Aggregate latency dashboards hide that gap. Measuring time to first token per model and per route is what tells a team whether a delay came from reasoning effort or from downstream work, and it is the metric a user actually experiences.
Copy link to heading4. Move the longest calls off the request thread
A high-effort call can outlast the execution window a route was configured for, and the timeout arrives after most of the model work is already done and paid for. Treat the longest reasoning work as a job rather than a request: stream from the first visible token, and queue anything that cannot finish inside the window.
The rest of what a reasoning route needs, long-running execution, spend enforcement, and provider failover, depends on the platform underneath it.
Copy link to headingHow Vercel powers reasoning model routing for engineering teams
Vercel puts model routing next to long-running execution and the observability that makes routing decisions measurable. A routing policy depends on all three: long calls need an execution window sized for them, per-request token data has to be available without custom instrumentation, and a degraded provider cannot be allowed to take the route down.
Copy link to headingAI Gateway routes requests between tiers
Teams without a dedicated ML infrastructure team tend to converge on one model for everything, because wiring per-request model selection into feature code never reaches the top of a sprint. The result is frontier pricing on classification traffic.
AI Gateway provides a single endpoint across providers, so the model a request uses becomes a string in runtime configuration rather than a provider integration in feature code. The cost-aware routing pattern documents the shape this takes in practice, classifying request difficulty, resolving a tier to a model string, and escalating only when a cheaper answer fails a verifiable check.
Gateway budgets are set per API key, and a key that exceeds its limit returns a 402 instead of passing the request through. Give each tier its own key, and a misrouting classifier produces rejected requests rather than a surprise invoice.
Copy link to headingFluid compute runs long reasoning calls
Intermittent timeouts on high-effort routes usually mean the execution window was inherited from an application default rather than chosen for what the route does. The thinking phase does not fit inside a window sized for a database query.
With Fluid compute, Vercel Functions default to 300 seconds on every plan and run up to 800 seconds on Pro and Enterprise. An extended maximum of 30 minutes is in beta for supported Node.js and Python runtime versions, set through function-level configuration. Because a reasoning call spends most of its window waiting on the provider rather than running code, the waiting time does not bill as active compute.
Work that needs no ceiling belongs in Vercel Workflows, which holds state across pauses without a duration limit.
Copy link to headingGateway observability shows what made a request slow
A slow request has two plausible explanations, model time or tool time, and telling them apart takes instrumentation that separates the two. Most teams build that view only after the first incident where nobody could say which one it was.
AI Gateway reports time to first token and token counts by model without additional instrumentation, available at both team and project level, which covers the reasoning-specific metrics that aggregate application dashboards hide. The Custom Reporting API breaks the same data down by model, user, tag, provider, or credential type when finer attribution is the question.
Vercel Drains streams logs and exports traces in OpenTelemetry format. Teams already running an observability backend get reasoning telemetry in the same place as their application and infrastructure traces, instead of in a separate tool.
Copy link to headingModel fallbacks keep a route serving
A reasoning request carries generated tokens and often tool state, so an error partway through costs more than a failed completion, and handling it in application code means provider-specific error branches piling up around the generation logic.
AI Gateway falls back to alternate providers automatically when the primary is unavailable, and a model fallbacks array covers the case where a model itself is unavailable. Provider ordering lives in application code, version-controlled alongside the generation logic:
typescript
import { streamText } from 'ai';
const result = streamText({ model: 'anthropic/claude-opus-5', prompt: 'Explain the tradeoffs of server-side rendering', providerOptions: { gateway: { order: ['vertex', 'bedrock', 'anthropic'], }, },});
for await (const chunk of result.textStream) { process.stdout.write(chunk);}
Ordering providers this way stops a degraded primary from taking the whole route down. Test that path on purpose, though, because an untested fallback is a config file, not a guarantee.
Copy link to headingRoute reasoning models by task on Vercel
The support router that pays reasoning prices on classification traffic and gains no accuracy has a routing problem, not a model problem: it runs a budget that scales with difficulty as though it scaled with traffic. Applying the reasoning-versus-standard choice per request is what separates a bill that tracks usage from one that tracks whatever users happened to send. Classify the task first, escalate on a defined signal, and measure the escalation's cost separately from the rest of the application.
Vercel provides the primitives that a routing policy depends on:
AI Gateway: One endpoint across providers, so escalation policy lives in runtime configuration rather than in feature code, with per-key budgets that reject over-limit requests instead of billing them.
Model fallbacks: An ordered provider list plus automatic failover, so a reasoning route keeps serving when the primary provider degrades.
Fluid compute: Execution windows sized for long reasoning calls, where provider wait time does not bill as active compute.
Extended function duration: 300 seconds by default on every plan, 800 seconds on Pro and Enterprise, and a 30-minute extended maximum in beta for long reasoning and tool calls.
Gateway observability and Vercel Drains: Time to first token and token counts by model at the team and project level, with OpenTelemetry export into the backend that the team already runs.
Deploy a routed AI feature by starting a new project, or browse Vercel templates for a working starting point.
Copy link to headingFrequently asked questions about reasoning models
Copy link to headingDo reasoning models always outperform standard LLMs?
No. Gains concentrate on multi-step math, code, multi-hop retrieval, and agentic planning. Standard models match them at lower cost on classification, extraction, and summarization, where extended reasoning adds latency and spend without adding accuracy.
Copy link to headingWhat is the difference between chain-of-thought prompting and a reasoning model?
Chain-of-thought prompting asks a standard model to show intermediate steps at inference time. A reasoning model is post-trained through fine-tuning, reinforcement learning, or distillation to generate and revise those steps by default, including the ability to abandon a path and try another.
Copy link to headingWhy is a reasoning model's bill higher than its visible output suggests?
Hidden reasoning tokens bill at the output rate even when they never appear in the response. Anthropic's Claude 4 models, for example, return summarized thinking output while billing the full thinking tokens, so visible length is a poor proxy for what a request costs.
Copy link to headingWhen should a request go to a standard model instead?
Route to a standard model when the task is single-step, latency-bound, or format-strict. Classification, extraction, summarization, and real-time chat are resolved through a shorter generation path, and standard models hold output schemas more reliably than reasoning models do.
Copy link to headingHow long should a reasoning call be allowed to run?
Long enough for the effort level the route sets, and no longer. On Vercel, functions default to 300 seconds with a maximum of 800 seconds on Pro and Enterprise, plus a 30-minute extended maximum in beta. Unbounded work belongs in Vercel Workflows.