An AI gateway sits between an agent and its models, while the Model Context Protocol (MCP) sits between the agent and its tools. Whether a stack needs both layers or just one starts with how much tool work its agents do.
On our AI Gateway network, the production index shows that tool work has overtaken plain inference, with tool calls now carrying the majority of token volume.
A tool-heavy workload can look like an argument for MCP over the gateway, but the two aren't substitutes. Vercel AI Gateway and MCP govern the same request path, just different halves of it. They overlap on the surface, since both route requests, handle authentication, and emit observability data, but AI Gateway acts on the model call while MCP acts on the tool call.
As tool work takes over, a serious agent can't run on the model plane alone. It still routes every model call through the gateway, but the tools it fires need governing too, which is why production agent stacks run both.
Copy link to headingKey takeaways
AI Gateway governs the model half of the request path and MCP the tool half. They're complementary, and most production stacks run both.
Both tools fit together in order. AI Gateway routes the model call, and, when the model decides to use a tool, MCP governs that call.
Tool-call tokens reached 58.9% of all tokens on our AI Gateway network by April 2026, up from 31.6% six months earlier, so tool governance now matters as much as model routing.
AI Gateway enforces model routing, failover, and spend controls. MCP servers enforce tool-level role-based access control (RBAC), argument-level policy, and the semantics of how tools execute.
The March 2025 MCP spec made Streamable HTTP the recommended transport, replacing Server-Sent Events (SSE), which changes how servers handle cross-call state on serverless infrastructure.
Both layers spend most of their runtime waiting on external systems, which is why Fluid compute's active-CPU billing pays off equally for AI Gateway and MCP servers.
Copy link to headingWhat is Vercel AI Gateway?
Vercel AI Gateway is your model-access plane, routing every request through a single endpoint to the right provider. The gateway standardizes requests across providers and centralizes the operational controls no single provider SDK gives you.
Five controls matter most:
Routing across hundreds of models by recent uptime and latency, with configurable order and fallbacks.
Automatic failover to the same model on another provider when one degrades, with no downtime.
Spend controls through per-key budgets that return a 402 when a key is exceeded, plus unified cost reporting.
Bring-your-own-key (BYOK) on the paid tier with zero markup on tokens.
Observability in the dashboard, with requests by model, time to first token (TTFT), token counts, and spend.
AI Gateway runs on Fluid compute, so you only pay for active CPU time. In its first month, 16,000 runtime hours included only 1,200 hours of actual CPU work, because the rest of the time it was waiting on model providers. Active-CPU pricing bills for that under 8% slice, so idle wait time costs you nothing.
Migrating to AI Gateway is a base-URL swap to https://ai-gateway.vercel.sh/v1, with no code changes for AI SDK v5 and v6, and OpenAI Chat Completions and Anthropic Messages clients point at the same endpoint.
Copy link to headingWhat is the MCP?
The MCP is an open standard for connecting AI applications to external tools and data. The protocol defines how a model discovers and invokes capabilities that live outside the model. Anthropic created the protocol and donated it to the Agentic AI Foundation under the Linux Foundation, where governance is community-driven.
The protocol runs on a client-host-server architecture. The host creates and manages multiple client instances, and each client holds an isolated one-to-one connection to a server. Servers expose Tools, the functions the model calls, over JSON-RPC 2.0 via stdio or Streamable HTTP, and the same protocol covers Resources, which are application-controlled data, and Prompts, which are user-controlled templates.
The spec is moving quickly, and the current production version, 2025-11-25, is already trailing a 2026-07-28 release candidate (RC) that is the biggest revision since MCP launched. The candidate makes MCP stateless at the protocol layer, so the Mcp-Session-Id header is gone, and a server that needs state across calls mints an explicit handle and has the model pass it back as a tool argument.
Our own MCP tooling covers building servers, hosting them, and consuming their tools:
mcp-handlerexports GET, POST, and DELETE handlers throughcreateMcpHandler(), supports Streamable HTTP and SSE, and wraps authentication withwithMcpAuth()for OAuth.mcp.vercel.comis our official remote MCP server with OAuth, supported in Claude, Cursor, ChatGPT, and VS Code.The AI SDK's
createMCPClientconverts MCP tools into AI SDK tools forgenerateText()andstreamText().
Teams like Zapier, Composio, Vapi, and Solana already run production MCP servers on Vercel, so the tooling is proven beyond internal use.
Copy link to headingHow AI Gateway and MCP divide the request path
AI Gateway and MCP sit on the same request path and see different parts of it. How they work together, and where they diverge, decides which layer owns a given problem.
Copy link to headingHow AI Gateway and MCP work together
AI Gateway and MCP run in sequence, with AI Gateway handling the model call and MCP taking over once the model decides to invoke a tool.
In Composio's framing, a large language model (LLM) gateway governs the "thinking traffic" into and out of the model, and an MCP gateway governs the "acting traffic" the model reaches once it calls a tool. Speakeasy draws the same line, with the LLM gateway in front of the model and the MCP layer after it decides to act.
Our own production architecture works the same way. We run the AI SDK, AI Gateway, and MCP servers as three distinct layers in one stack, so model-access decisions and tool-execution decisions never collide.
Copy link to headingHow they differ across the dimensions that matter in production
The clearest way to see the split is by what each layer can watch. AI Gateway sees prompts, completions, providers, tokens, and spend. MCP sees tool names, arguments, responses, and tool latency. The two watch different traffic, so they break down differently.
Auth and failure modes separate them most. AI Gateway's failures are provider outages, rate limits, and quota breaches, and they're handled with routing and budgets. MCP's failures are tool poisoning, confused-deputy attacks, and lost session state, and they're handled with tool-level policy and authorization.
Copy link to headingWhen to use AI Gateway, MCP, or both
Which layers you need follows from what your application does.
Copy link to heading1. Use AI Gateway when you make direct model calls with no agent loop
Reach for AI Gateway alone when the application calls models directly and runs no agent loop. Summarization, classification, and single-provider generation pipelines fit here, especially the ones that still need multi-provider failover and spend controls without any tool execution. With no tools to invoke, there's nothing for MCP to govern, and the model-access plane is the only layer the workload needs.
Copy link to heading2. Use MCP when you're small-scale and single-provider
MCP servers without a dedicated AI gateway make sense for a small, single-provider application where failover and spend controls aren't requirements yet.
Without a gateway, though, every agent manages its own credentials, rate limits, and tool definitions, and, past a handful of connected servers, the surface area gets unmanageable.
That’s the point where an AI gateway earns its place alongside MCP.
Copy link to heading3. Use both when you run an agent loop at scale
Run both once the application drives an agent loop with tool calls and needs multi-provider routing. Per-entity spend limits and tool-cost attribution live here too. Our own three-layer stack is the reference shape. The AI SDK's ToolLoopAgent handles the agent abstraction, AI Gateway handles model routing and spend, and MCP servers handle tool execution. The gateway's tags field carries cost attribution across both layers.
The tool layer is where the biggest gains hide at this scale. When we removed 80% of an internal agent's tools, three things improved at once:
Average execution time dropped from 274.8 seconds to 77.4 seconds, 3.5 times faster
The success rate went from 80% to 100%
Token usage fell 37%, from roughly 102,000 to 61,000 tokens per run
That agent ran Claude Opus 4.5 through the AI SDK, with Vercel Sandbox and AI Gateway underneath, so which tools it exposed moved cost and reliability more than any model-routing change did.
One deployment caveat applies whichever scenario you land in. Install mcp-handler with @modelcontextprotocol/sdk@1.26.0 or later, since earlier versions carry a documented security vulnerability, and track MCP release updates, because the removal of Mcp-Session-Id affects any server that relies on session continuity.
Copy link to headingRun both planes on Vercel
The request path for a production agent runs through two planes, a model-access plane and a tool-execution plane. On our AI Gateway network, tool-call requests grew from 31.6% to 58.9% of tokens between October 2025 and April 2026, so most token volume now comes from agents that call tools.
Governing the model plane without governing those tool calls covers only part of what an agent does, and that gap is what MCP fills. The question was never AI Gateway or MCP, but whether you govern both halves of the request.
Production teams already run the two planes side by side. Cloudflare's internal AI engineering stack pairs its AI Gateway for centralized model routing, cost tracking, and BYOK with a separate MCP Server Portal for tool access, keeping the two distinct.
On Vercel, both planes run as managed infrastructure:
Automatic failover: AI Gateway reissues a failed request to a healthy provider, which is how 3.5% of fleet requests still succeed.
Zero-markup pricing with BYOK: Tokens bill at the provider's list price, including when you bring your own key.
Spend controls: Per-key budgets return a 402 when a key is exceeded, so a runaway agent loop becomes a rejected request instead of a surprise invoice.
Built-in observability: Per-model TTFT, spend, and token counts report in one view, with a Custom Reporting API queryable by user, tag, provider, or credential type.
Governed MCP servers:
mcp-handlerandmcp.vercel.comdeploy the tool plane with OAuth and Streamable HTTP support.
Start a new project at vercel.com/new and route your first model call through AI Gateway, then add an MCP server with mcp-handler when your agents need governed tools. The AI Gateway docs cover the model plane end-to-end.
Copy link to headingFAQ about AI Gateway vs MCP
Copy link to headingCan I use Vercel AI Gateway as my MCP gateway?
No. AI Gateway handles model routing, failover, spend controls, and observability for inference calls. Tool-level RBAC, argument-level policy, and MCP authorization semantics belong in the tool-execution layer. Our MCP tooling, mcp-handler and mcp.vercel.com, covers that layer separately. Run both in the same stack, because neither substitutes for the other.
Copy link to headingDoes the 2026-07-28 MCP spec RC change how I should deploy MCP servers on Vercel?
Yes, if your server depends on session state. The release candidate removes the Mcp-Session-Id header and the protocol-level session, so any request can land on any server instance. A server that still needs state across calls mints an explicit handle from a tool and has the model pass it back as an argument on later calls. Pin @modelcontextprotocol/sdk to a version that supports 2026-07-28, and watch the upstream release notes.
Copy link to headingWhy does tool-call token share matter for infrastructure decisions?
Token cost is the main operating expense for agentic applications, and tool-using requests now carry the majority of those tokens, as our production index shows. Which tools an agent can reach, with what arguments, and under whose credentials, shapes cost and reliability as much as model routing does, which makes tool governance a first-class concern rather than an afterthought.
Copy link to headingWhat is the Fluid compute advantage for running both AI Gateway and MCP servers?
Both workloads are I/O-bound. AI Gateway spends most of each request waiting on inference providers at under 8% CPU utilization, and MCP servers spend most of their execution waiting on external APIs and databases. Active-CPU billing means neither pays for idle wait time, and one team measured over 90% savings versus traditional serverless on exactly that basis.