AI Gateway Observability and Spend
AI Gateway logs every request and gives you the tools to understand and control what you spend. You can monitor latency and token usage in the dashboard, query spend by model, user, or tag, track credit balances, and cap how much your team, a project, or an API key can spend with budgets.
- Monitor every request: Track latency, token counts, and spend in the dashboard with Observability
- Inspect individual requests: Search, filter, and export request logs to debug routing and errors
- Query usage data: Break down costs by model, user, tag, or provider with the Custom Reporting API
- Look up one generation: Retrieve its provider, latency, token usage, cost, and finish reason with the Generation Lookup API
- Cap spend: Set a spending limit for your team, a project, or an API key with Budgets
- Export traces: Forward an OpenTelemetry trace of every request to your own observability tool with Trace Drains
| Feature | What it does | Key details |
|---|---|---|
| Observability | Monitor and debug AI requests | Request traces, token counts, latency metrics, spend tracking |
| Custom Reporting | Query usage data with flexible grouping | Group by model, user, tag, provider, or credential type; filter by date |
| Generation Lookup | Inspect one generation by ID | Provider, latency, tokens, cost, finish reason, and credit balance API |
| Budgets | Cap spending by team, project, or key | Scoped limits, daily/weekly/monthly refresh, spend alerts, defaults |
| Trace Drains | Export request traces via OTLP | OpenTelemetry traces, provider-attempt spans, per-trace-event billing |
AI Gateway automatically logs every request with metrics you can view in the Vercel dashboard:
- Requests by model: See which models your application uses most
- Time to first token (TTFT): Monitor response latency
- Token counts: Track input and output token usage
- Spend: View costs broken down by model and time period
Access these metrics from the Observability tab at both team and project levels.
The Custom Reporting API lets you break down costs and token consumption by model, user, tag, provider, or credential type. Filter by date range, specific users, models, and tags to understand exactly where your AI spend is going.
curl "https://ai-gateway.vercel.sh/v1/report?start_date=2026-01-01&end_date=2026-01-31&group_by=model" \
-H "Authorization: Bearer YOUR_API_KEY"Attach user and tags to your requests to enable per-user and per-tag reporting. See the Custom Reporting docs for the full API reference.
Give your team, a single project, or an individual API key a budget to cap how much it can spend. AI Gateway checks the budget before each request and stops further requests once the limit is reached, with optional daily, weekly, or monthly refresh periods. See the Budgets docs to set one up.
- View your observability dashboard to monitor usage
- Query usage data with the Custom Reporting API
- Look up a generation by ID, or check your credit balance
- Set a budget to cap spending
Was this helpful?