Teams shipping reliable AI products today have almost nothing in common at the model layer. Some run Claude, some run GPT, some run three providers at once and couldn't say offhand which one served the last request. What they share is upstream of the model: they treat it as a swappable component behind a routing layer they own, not as a decision made once and built around.
That's the inversion this piece is about. Most rankings of the best AI models for developers fixate on which model wins this month. The model you pick matters less than how you architect around it, and the data backs that up. Routing billions of tokens through our AI Gateway has made one thing clear: the model is a config value with a shelf life of about a quarter. The routing layer is the asset that lasts. The rest of this is the case for that claim, walked through the six places the decision actually gets made: how you benchmark, how you fall back, how you route by cost, how you abstract the provider, how you run the infrastructure underneath, and how you evaluate. They are not six practices. They are one architectural stance applied six times.
Copy link to headingHow to choose the best AI models for developers
The useful question is not which model tops the leaderboard this month. It's which model for which request, under which constraints. The field has split by role: low-cost, fast models for routine work, and frontier models for the requests that are ambiguous, multi-step, or expensive to get wrong. The price gap between those roles is wide enough that the routing decision, not the picking decision, is where the leverage sits.
Read the table by role, not by rank. The frontier models earn their price on hard reasoning and high-stakes generation. The low-cost models handle the routine majority of requests at a fraction of the cost. The architecture that lets you send each request to the right row is the rest of this piece.
Copy link to headingBenchmarks stopped predicting production performance sometime in 2025
Start model selection by checking the leaderboards and you have already made your first mistake. The leaderboard measures a model against a test set. You are shipping it against your workload. Those are different things, and the gap between them stopped being academic about a year ago.
This showed up directly when frontier models were evaluated for v0's code generation pipeline. On the same task, our v0 composite model evaluation found v0-1.5-md achieved 93.87% error-free generation, while claude-4-opus hit 78.43%, and gemini-2.5-pro-preview-05-06 and o3 both landed at 58.82%. Frontier general-purpose models scored 15 to 35 points below the task-specific one. No public leaderboard predicted that, because no public leaderboard was running our task.
This is not a v0 quirk. Models that post 84–89% on synthetic benchmarks attain only 25–34% on real-world class-level tasks. LXT's benchmark analysis found a 13% accuracy drop on GSM8K once contaminated examples were removed, which tells you contamination is a variable to measure, not a footnote to wave away. The score you read on a leaderboard has had its hardest cases memorized out of it.
So when v0's evaluation suite was built, it started with hundreds of domain-specific examples drawn from the work the model would actually do. That is the only benchmark that has ever predicted production for this platform. The leaderboard is a starting rumor, not evidence. Choosing a model today on a number someone else generated against a task that isn't yours means optimizing for a workload you don't run.
Copy link to headingRouting is the architecture, not a feature you bolt on later
The single-provider integration is the most expensive cheap decision in AI engineering. It looks free at the start, because hardcoding one provider's SDK is the fastest way to a working demo. The bill comes later, and it comes twice: once when a better model launches and you need weeks to rewrite integration code to reach it, and once when your provider has an incident and you have no second path to fail over to. We built the AI Gateway because we kept watching the same teams pay both bills, while the teams that abstracted the model layer from day one absorbed the same events in hours.
The fallback data is the part people underestimate. Through April 2026, our Gateway rescued 3.5% of requests, 5.1% of tokens, and 4.9% of market cost through automatic fallback. Read 3.5% as edge-case noise and you've missed it. For a team running millions of requests a month, that is the line between a product that stays up and a product that goes down every time one provider does. And providers do go down: Maxim's read of status pages counted 20 incidents from Anthropic and 22 from OpenAI in a single month, December 2025, some of them major outages past the 30-minute mark. You are not choosing whether your provider will fail. You are choosing whether you've built the second path before it does.
The shift in what the traffic is made of makes this sharper. Tool-call share of Gateway tokens rose from 31.6% in October 2025 to 58.9% in April 2026, and tool-call share of requests went from 11.4% to 22.2% over the same window. Agentic workloads are now the majority of tokens in that slice, and a single user request can fan out into repeated model invocations across tool calls and reasoning steps. When one request depends on ten model calls, the probability that all ten land cleanly on a single provider is not a rounding error you can ignore. Routing stops being optional the moment your requests stop being atomic.
The practical stance follows from the data. Plan for multiple models across providers, assume you'll need fallbacks, and treat routing as a core unit of architecture from the first commit rather than a resilience feature you add after the first outage. For teams handling sensitive data, the Gateway supports filtering to only providers with zero data retention, then sorting by time to first token, so the abstraction buys you a compliance lever, not just a reliability one.
Copy link to headingMost of your requests don't need the frontier model
The production evidence across our own templates, team deployments, and cost analyses points the same direction: most teams overspend on frontier models because they route every request to the same expensive endpoint. Sending a one-line classification task and a multi-step reasoning task to the identical model is a common pattern, and it's why inference bills scale faster than usage. The fix is not a cheaper model. It's a router. Classify complexity first, send routine work to lower-cost models, and reserve frontier models for the requests that actually need them.
The shape of the pattern is consistent enough to draw:
The pricing spread that makes the routing worth the wiring is the one in the table above: frontier models like GPT-5.6 Sol and Claude Opus 4.8 run $5.00 per million input tokens against $25–30 on output, while low-cost options like Gemini 3.1 Flash-Lite and DeepSeek V4 Flash land well below that. That is a wide gap between the top and bottom of the stack, and most requests do not need the top. MorphLLM's cost model estimates that a router classifying request difficulty and sending easy tasks to Haiku 4.5, medium to Sonnet 5, and hard to Opus 4.8 reaches 40–70% typical cost savings because 60–80% of requests are routine. The tradeoff is real and worth naming: a cheaper model will get some requests wrong that the frontier model would have gotten right, so the classification step is where the risk lives. Get the classifier wrong and you've traded cost for quality on the requests that mattered most. The win is not "use cheap models." It's "spend your capability budget where the hard requests are."
This shows up in how teams use the AI Gateway. Our Savoir template includes a smart complexity router that classifies incoming questions and routes them to the right model, simple questions to fast, cheap models and hard questions to powerful ones. SERHANT., a real estate technology company, built S.MPLE on Vercel to orchestrate OpenAI, Claude, and Gemini by task to optimize cost against output. The result they care about: they scaled from 200 to 800–900+ agents without replatforming, because task-based routing kept the unit economics viable as the agent count grew. That is the compounding payoff. Routing doesn't just lower today's bill, it keeps the bill from becoming the reason you can't scale.
The savings stack, too. Prompt caching cuts repeated-context costs on providers that support it. Layer that with batch processing discounts of 50%, and DigitalApplied's cost model argues that rack-rate forecasting overstates actual LLM spend by 3–10x once cache, batch, and reserved discounts enter the picture. The team budgeting off sticker price is planning for a bill they will never receive, and usually over-provisioning the model tier to match.
Copy link to headingProvider abstraction pays for itself the first time you switch
The first switch costs more than expected when the abstraction isn't there, and it's rarely the two lines a team hoped for. Thomson Reuters learned the same lesson building CoCounsel, their AI assistant for attorneys and accountants. Three developers built it in two months using the AI SDK. The team is now serving 1,300 accounting firms and migrating their entire codebase to the AI SDK, deprecating thousands of lines of code across 10 providers and consolidating into one composable system. Ten provider integrations collapsing into one is not a tidy project. It's the difference between a switch being a configuration change and a switch being a quarter of engineering work.
The AI SDK reduces provider switching to that configuration change. Each provider has its own method for interfacing with its models, which is what makes switching expensive and lock-in quiet. The SDK abstracts those differences into a unified interface, so swapping Anthropic's Claude for OpenAI's GPT-5.6 takes changing two lines of code. Structured output is the case that makes it concrete: AI SDK Core exposes one API for generating structured objects, so you don't maintain a separate code path per provider for the same application behavior. The honest cost of any abstraction is that it tracks the lowest common denominator. A unified interface can lag a provider's newest capability until the SDK exposes it. In practice, the lag is measured against the alternative, which is rewriting your integration by hand every time, and the abstraction wins that comparison nearly every time.
Zo Computer shows what the abstraction buys at scale. 91.88% of their traffic routes through Vercel. They handle 3.3x larger context windows, 42,500 average input tokens against 12,700, at a lower error rate than their non-Vercel path, with 99% of requests completing in under 81 seconds. The number that tells the architectural story: new model support dropped from an hour-long, multi-file code change to a far faster process. When adding a model costs minutes instead of an afternoon, you add models you would otherwise have skipped.
Gamma's image pipeline makes the same point from a different angle. They've generated more than 1.5 billion images across 60 models and 20 providers. When the AI SDK introduced ImageModelV3, Gamma's team could integrate new models within days of launch. That speed compounds: days instead of weeks, across 60 models, over the life of the product. The per-integration saving looks small until you multiply it by the number of integrations a serious AI product actually does.
As one team put it: "If the team picked one frontier model and built tightly around it, every future change would come with a rewrite, with no clean path to fallback when reliability or cost shifted. With AI SDK, iteration meant simple configuration changes, not feature overhauls."
Copy link to headingLLM workloads break traditional serverless, and the fix is architectural
LLM workloads spend most of their wall-clock time waiting. Your function sends a prompt, waits seconds for the model to respond, then streams tokens back. Those wait times and long idle periods break the operational model of traditional serverless, which was built on the assumption that a function is either working or finished, not parked waiting on someone else's GPU. AI gives serverless a third state it was never priced for, and traditional billing charges you for all of it. We built Fluid compute to answer exactly this, after watching wait-heavy AI workloads turn a billing model that worked fine for request-response apps into a tax.
There are three taxes traditional serverless levies on AI workloads, and the architecture has to answer each one:
You pay while you wait. Fluid compute charges for CPU only while it is actively doing work, so time spent waiting for database queries, API responses, or model inference does not count toward compute costs. For a workload that is mostly waiting, that is the line between viable unit economics and a cost structure that scales against you.
You pay per instance. Fluid compute orchestrates compute across invocations, letting multiple concurrent requests share the same underlying resources instead of spinning up a separate instance per request. Teams saw up to 85% cost savings through in-function concurrency, which is exactly the saving an I/O-bound, wait-heavy workload should be able to capture.
You pay the cold-start latency. 99.37% of all requests on paid plans see zero cold starts, which removes the other serverless tax that hits AI apps hardest, the one users feel as a stall before the first token.
Streaming is the last piece, and it changes the user-experience math more than the cost math. Traditional serverless functions send responses all at once, but Vercel Functions deliver data in chunks as it becomes available. For responses that take 5–30 seconds to deliver, showing partial results immediately is the difference between an app that feels alive and one that feels broken. The AI SDK's streamText function and useChat React hooks handle the client-side complexity, and data parts give you a first-class way to stream arbitrary, type-safe data from server to client. The cost of streaming is that your client code now reasons about partial state instead of a single response object, and for any AI product worth shipping, that's a cost worth paying.
Copy link to headingBuild evaluation first, or you're routing blind
The strongest models in production didn't get there by being the strongest. They got there because the team built evaluation before they built anything else, so they could tell, on their own data, which model was actually winning. Routing without evaluation is just guessing with extra steps. Here's the lifecycle that works across our platform and the one we run internally.
Domain-specific benchmarking comes first. The teams shipping reliably run candidates against their own test sets, hundreds of examples weighted by use case, not someone else's leaderboard. Gigenet frames the tradeoff that matters: a model that gives excellent results 80% of the time but fails dramatically in the other 20% may be worth less than one with slightly lower peak performance and more consistent results. Consistency is a feature, and the only way to see it is to measure the tail, not the average.
Shadow deployment and continuous monitoring come next. Run the candidate in parallel with production, collecting latency percentiles, accuracy scores, and cost per request. The Gateway routes a slice of traffic to the candidate while the primary model still serves users, so you measure on real requests without betting users on an unproven model. Then continuous monitoring catches the silent degradation, because every tool call, reasoning step, and token count should be traceable so you can diagnose failures and optimize performance. We provide built-in observability, and the AI SDK integrates with Grafana AI Observability, Datadog LLM Observability, and OpenLIT via OpenTelemetry. The reason this is not optional: model providers update or deprecate versions without notice, changing outputs in ways that break workflows. Drift detection across accuracy, latency, cost, and safety is ongoing, not a launch checklist.
Feedback loops close it. Production errors become test cases. Vellum describes the loop in operational terms: a confusing query that stumps a chatbot becomes a new test case, and recurring hallucination types inform prompt adjustments or fine-tuning. This is where evaluation stops being a phase and starts compounding, because every failure captured makes the next model decision better-informed than the last.
The sequencing is the actual advice. We deploy observability before optimizing anything else, because the classic mistake is bolting it on after the first incident, by which point the team is debugging a degradation it can't see and can't reproduce. By the time observability feels necessary, the point where adding it was cheap has already passed.
Copy link to headingWhere routing itself breaks down
Routing isn't free. Pushing the model decision out of the codebase and into a layer buys flexibility, and it costs simplicity. A team that adopts the whole stance without accounting for that cost will be surprised by it.
The complexity is real. A router, a fallback policy, a complexity classifier, and a shadow-deployment pipeline are four moving parts that a hardcoded single-provider integration doesn't have, and each is a new surface that can fail. The classifier is the sharpest edge: route a hard request to a cheap model because the classifier misjudged it, and you've shipped a worse answer to the request that mattered most, silently. The abstraction layer has its own tax, tracking the lowest common denominator across providers until the SDK catches up to a new capability. And evaluation is not a setup cost you pay once. Shadow deployments mean two versions running at the same time, which you have to keep cleanly separated, and drift detection runs for as long as the product lives.
None of this argues against routing. It argues for adopting it deliberately, in the order that pays off soonest. Abstraction and fallback earn their keep almost immediately, on the first provider incident. Complexity routing earns its keep once volume is high enough that the cost gap is real money. Full shadow-deployment evaluation is the heaviest investment and the last to pay back, so it's the piece to grow into rather than front-load. The teams that get burned are the ones who treat the whole stance as all-or-nothing instead of sequencing it against their own scale.
Copy link to headingRouting replaces picking
Model selection in 2026 is a routing problem, not a picking problem. The frontier is close enough at the top that picking barely matters: current benchmarks put GPT-5.6 and Claude Opus 4.8 within a fraction of a point of each other on several agentic and reasoning tasks. When the best models are within a point of each other, the model you chose is not what separates a production-ready AI product from a fragile prototype. The infrastructure around the model is: provider abstraction, automatic fallback, complexity-based routing, streaming architecture, and continuous evaluation.
Look back across those six sections and they're all the same move. Benchmarking on your own data, falling back across providers, routing by complexity, abstracting the provider, running wait-aware infrastructure, evaluating continuously, none of them is about finding the right model. Every one is about making the model decision cheap to revisit, so that when the right model changes, and it will, the change is a config edit instead of a rewrite. The teams that "picked the right model" didn't win. The teams that made the choice disposable did.
We built these tools because we needed them ourselves. We are going through the same AI transformation as the teams building on us, and the AI SDK now sees more than 16 million downloads a week carrying that pattern into production. The pattern is the whole point: abstract the model layer, route by task complexity, measure everything, and design from day one for the switch you will inevitably make.
Copy link to headingFAQs
Copy link to headingWhat are the best AI models for developers right now?
It depends on the request, which is the whole point. For hard, reasoning-heavy work, frontier models like GPT-5.6 Sol and Claude Opus 4.8 are worth their price. For the routine majority of requests, low-cost models like Gemini 3.1 Flash-Lite and DeepSeek V4 Flash do the job at a fraction of the cost. The strongest setup isn't one winning model, it's a router that classifies each request and sends it to the right tier, behind an abstraction layer so you can swap any of them as the field moves. Our own v0 evaluation found a task-specific model beating top general-purpose models by up to 35 points on the same task, which is why "best" only means anything relative to your workload.
Copy link to headingHow much can routing by complexity save on LLM costs?
MorphLLM's cost model estimates 40–70% typical savings, because 60–80% of requests are routine and don't need a frontier model. The mechanism is simple: classify each request's difficulty, send easy requests to a low-cost model and hard ones to a frontier model, instead of paying the top rate for everything. The savings compound with prompt caching and batch discounts, to the point that DigitalApplied's model argues rack-rate forecasting overstates real spend by 3–10x once those enter the picture. The risk to manage is classifier accuracy, since a misrouted hard request gets a worse answer.
Copy link to headingWhat is an AI gateway, and why use one?
An AI gateway is the routing layer between your application and the model providers. Instead of calling one provider's API directly, you call the gateway, and it handles provider selection, automatic fallback when a provider fails, and routing rules like complexity-based tiering or filtering to zero-data-retention providers. The value shows up under failure: through April 2026, our AI Gateway rescued 3.5% of requests, 5.1% of tokens, and 4.9% of market cost through automatic fallback, against a backdrop of 20 Anthropic and 22 OpenAI incidents in a single month.
Copy link to headingHow do I switch LLM providers without rewriting my app?
Use an abstraction layer that gives every provider a unified interface. With the AI SDK, swapping Anthropic's Claude for OpenAI's GPT-5.6 takes changing two lines of code, because the SDK abstracts each provider's distinct API into one shape, including structured output. Thomson Reuters took this to its conclusion on CoCounsel, consolidating thousands of lines across 10 provider integrations into one composable system. The tradeoff is that a unified interface can lag a provider's newest feature until the SDK exposes it, which is nearly always cheaper than maintaining per-provider integration code by hand.