Your storefront has a decision to make every time a customer submits a review. Before it appears under the product, something has to decide whether it's spam, whether it exposes personal information, and which section it belongs in. Keyword rules can't tell a shipping complaint from a product defect, or a discount code in spam from one a customer mentions in passing. Asking a language model means waiting on generated text and parsing it before your code can act.
Jev, a System One model from TypeSafe AI, evaluates supplied state against typed questions and returns choices, scores, and boolean probabilities without generating prose. TanStack AI exposes those answers through its decide() function, and the @tanstack/ai-vercel-gateway adapter sends each call through Vercel AI Gateway, so evaluation requests share credentials, logs, budgets, and data controls with the rest of your Gateway traffic.
Your application decides what happens next, so a topic choice can place a review in the right section, a sentiment score can feed a rating summary, and a spam flag or uncertain answer can hold the review for a moderator. Keeping those rules in code lets you change how the application responds without redefining what you ask the model to assess.
Copy link to headingOverview
In this guide, you'll learn how to:
- Ask a single yes-or-no question about a piece of state
- Answer several typed questions in one request from a TanStack Start app
- Branch on probability and confidence so clear reviews publish automatically and flagged or uncertain ones go to a moderator
- Unit-test that branching without calling Jev
- Set Gateway data controls, cancel in-flight requests, and track token usage
Copy link to headingPrerequisites
Before you begin, make sure you have:
- A Vercel account
- Vercel CLI installed (
npm i -g vercel) - Node.js 22+ and a package manager (e.g., pnpm)
- An existing TanStack Start project
Copy link to headingHow TanStack AI evaluates with Jev
TanStack AI's decide() function takes an adapter that names the model and provider, a state to evaluate, and a map of questions built with the choice(), score(), and boolean() helpers. Unlike chat(), it doesn't stream. The call resolves to one object with a typed answer for every question key.
The @tanstack/ai-vercel-gateway adapter provides vercelGatewayDecider, which sends the request to AI Gateway's evaluation endpoint.
TanStack AI also ships an adapter that calls TypeSafe directly. Both implement the same evaluate activity, so switching means changing the adapter and nothing else.
Copy link to headingThree question helpers
Each TanStack AI helper corresponds to a Jev question type. Answers appear directly on the result under their question keys, so questions.topic becomes result.topic.
The meta key holds usage and model information and cannot be used as a question name.
| Helper | What it does | Configuration | Answer fields |
|---|---|---|---|
choice() | Picks one option from a named set | options: map of option keys to descriptions (null for none) | type, value, probability, probabilities, confidence |
score() | Grades the state against an ordered rubric | levels: array of at least two labels, lowest to highest | type, value, score, probability, probabilities, legend, confidence |
boolean() | Estimates whether a statement is true | Optional criteria: { true, false } descriptions | type, value, probability |
Reading the answer fields:
typeis'choice','score', or'boolean', so you can narrow on it when handling answers generically.valueis the decision. Forchoice(), it's the selected option key, typed as a union of your keys. Forscore(), it's the label of the level nearest toscore. Forboolean(), it'struewhenprobabilityis 0.5 or higher.probabilitydescribes the returnedvaluefor choice and score answers. For boolean answers, it estimates whether the statement is true, so0.98is a strong yes and0.02is a strong no.probabilitiescontains the full distribution, using option names for choice answers and level indices for score answers. The score answer'slegendmaps those indices to labels.scoreis a fractional position on the level scale, indexed from zero.1.08falls between the second and third levels, andvaluenames the nearest.confidenceis TypeSafe's statistic for how concentrated the distribution is, from0(spread evenly) to1(all on one answer). It's returned forchoice()andscore()answers, not forboolean().
The AI SDK's experimental_evaluate returns answers under result.answers and TypeSafe's confidence statistics in providerMetadata. TanStack AI's decide() places answers directly on the result, with confidence included on choice and score answers.
You can adapt the support-ticket example in the AI SDK guide to compare how the two APIs express the same routing decisions.
Copy link to headingSteps
Copy link to heading1. Install TanStack AI and the AI Gateway adapter
Install the core package and the adapter:
pnpm i @tanstack/ai @tanstack/ai-vercel-gatewaynpm i @tanstack/ai @tanstack/ai-vercel-gatewayyarn add @tanstack/ai @tanstack/ai-vercel-gatewaybun add @tanstack/ai @tanstack/ai-vercel-gateway@tanstack/ai exports decide() and the question helpers.
@tanstack/ai-vercel-gateway exports vercelGatewayDecider along with the chat, embedding, image, and summarize adapters covered in Using TanStack AI with Vercel AI Gateway.
Copy link to heading2. Authenticate with AI Gateway
The adapter reads AI_GATEWAY_API_KEY from your environment. When that variable isn't set, it falls back to VERCEL_OIDC_TOKEN. Deployments on Vercel receive an OIDC token automatically, so no key is needed in production.
For local development, connect to AI Gateway using one of the following methods. You can pull an OIDC token from your linked Vercel project or add an AI Gateway API key to .env.local.
Copy link to headingOption 1: Vercel OIDC
Link the directory to your Vercel project and pull its development environment variables. This writes a VERCEL_OIDC_TOKEN to .env.local:
The local token expires after 12 hours. Re-run vercel env pull .env.local when a request returns an unauthorized error.
Copy link to headingOption 2: AI Gateway API key
API keys work anywhere, including CI and servers outside Vercel. Create a key in the Vercel dashboard and set it as AI_GATEWAY_API_KEY in .env.local:
To pass a key explicitly instead, use the create* factory. This is useful when the key comes from a service like Vercel Connect rather than an environment variable:
Whichever option you choose, keep the credential on the server. decide() runs in server code and is called from the browser over fetch.
Copy link to heading3. Ask one boolean question
Start by asking whether a customer review contains promotional content. Pass the review text as state and name the question isPromotional to read its typed answer from result.isPromotional.
For a review like 'Skip this one. Get the real thing 40% off at dealz-outlet dot com with code SAVE40.', expect isPromotional() to return a probability close to 1, with result.isPromotional.value set to true.
The criteria on a boolean question are optional, but they sharpen the decision by telling the model exactly what counts as true and false.
Copy link to heading4. Answer several questions against structured state in one request
Jev evaluates the questions in a request independently and in parallel. You can combine choice(), score(), and boolean() questions against the same state.
Pass a string, object, or array as state without serializing it yourself. An array, such as a message history, provides one shared context for all questions; unrelated inputs need separate evaluations.
On a headless storefront, reviews can reach your server route from a custom form built with Vercel Shop or Hydrogen, where Shopify or another commerce backend serves the product data, or through a webhook from a reviews app.
The following TanStack Start server route reuses an adapter created at module scope to evaluate each review's topic, sentiment, and two content flags in one call.
TanStack Start exposes src/routes/api.moderate.ts at /api/moderate.
parseReview checks the four fields Jev will see and rejects anything else with a 400. For a production form, a schema library such as Zod does the same job with less code. Whether the reviewer is a verified purchaser belongs to your commerce backend, not the request body, so look it up there when your publish rules need it.
The supplied state includes the customer's star rating, while the sentiment question asks Jev to assess the review text. Comparing them can reveal mismatches, such as a five-star review describing a poor experience with the product.
Start the dev server and send a product review:
pnpm devnpm run devyarn devbun devThe response contains a typed answer under each question's key, with the resolved model and token usage in meta. Jev reports completionTokens but bills only for input tokens. The example below shows possible values for this review, which the policy in step five maps to a moderation decision using fixed thresholds:
Reading the response:
topic.valueis typed from the declared option keys, so TypeScript flags comparisons with an unknown option, such asvalue === 'quality'.topic.probabilitiescovers every option, and the selected value always has the highest probability. Here the review mentions shipping, and the distribution reflects that without changing the answer.sentiment.valuenames the level nearest tosentiment.score, which isn't always the level with the highest probability.sentiment.legendmaps the index keys insentiment.probabilitiesback to their labels.isPromotional.probabilityis the estimated probability that the review is promotional, not a confidence in the answer.
Copy link to heading5. Branch on probability and confidence
Use the returned statistics to decide which reviews can publish automatically and which need human review. Set stricter thresholds where a wrong decision has greater consequences.
Publishing spam or someone's phone number is costly and holding a review for a moderator is cheap, so the two flags use a low floor while the topic gate is stricter.
The moderation policy below uses three paths:
| Condition | What your code does |
|---|---|
isPromotional or mentionsPersonalInfo probability is ≥ 0.3 | Hold the review and record which flag tripped |
Topic confidence is below 0.6, the selected option's probability is below 0.7, or the topic is notAReview | Hold the review as unclear or off-topic |
| Otherwise | Publish the review in the section for its topic |
Keep the moderation policy in a pure function that accepts the evaluation result. Separating it from decide() lets the tests in step six check policy decisions using fixed answer objects, without calling Jev.
Then update the POST handler from step four to return the moderation decision instead of the raw answers:
Sending the same curl request from step four now returns a decision:
action: 'publish' means the evaluation passed the moderation thresholds above. Whether the review goes live still depends on rules Jev doesn't see, such as a verified-purchase requirement, per-account review limits, or a product that has been delisted, so keep those as a second check in your code.
Treat the example thresholds as starting points. Calibration describes how predicted probabilities match observed outcomes across many examples; it doesn't guarantee an individual answer is correct. Run labeled reviews through the same questions, then choose cutoffs based on the errors your workflow can tolerate.
Copy link to heading6. Test the moderation policy without calling Jev
Because moderateReview takes a result object rather than a review, you can test each branch with fixed answers and no network call.
Add Vitest as a development dependency:
pnpm add -D vitestnpm install -D vitestyarn add -D vitestbun add -d vitestThe clearAnswers fixture is typed against ReviewEvaluation, so TypeScript catches an answer shape that drifts from the questions in step four. Each hold test isolates one condition, so a test fails if that condition is removed from the policy, and the boundary tests pin the 0.3, 0.6, and 0.7 cutoffs.
Run the tests:
pnpm vitest run src/lib/moderate-review.test.tsnpx vitest run src/lib/moderate-review.test.tsyarn vitest run src/lib/moderate-review.test.tsbunx vitest run src/lib/moderate-review.test.tsExpected output:
Testing the thresholds this way is separate from checking whether the thresholds are right for your data. For that, run real reviews with known moderation outcomes through Jev and compare its probabilities to what happened.
Copy link to heading7. Set Gateway options, cancel requests, and track usage
Alongside adapter, state, and questions, decide() accepts options for data controls, cancellation, and monitoring:
- Set
modelOptions.gatewayto apply controls such as Zero Data Retention and No Training. The adapter forwards these settings asproviderOptions.gateway. - Pass an
abortSignalto cancel an in-flight request when your application aborts it or a timeout expires. - Add observe-only
middlewareto track completion, errors, and cancellation. ItsonUsagehook receives the token counts for each call.
Evaluation requests also appear in AI Gateway logs and count toward budgets.
Jev charges for input tokens only, so promptTokens is what you're billed for. To emit OpenTelemetry spans for evaluate calls instead of logging, pass TanStack AI's otelMiddleware() in the same middleware array.
Copy link to headingOther storefront decisions that fit a typed question
Other e-commerce tasks also involve interpreting customer requests or product information to choose from a defined set of outcomes.
The examples below use Jev to assess the supplied record, while application code applies thresholds and business rules to decide what happens next.
| Decision | State | Questions | Your code then |
|---|---|---|---|
| Route a return request | Order details and the customer's message | choice() for reason (defective, wrong item, changed mind, not received); boolean() for whether the item is described as used | Auto-approve within policy, otherwise queue for an agent |
| Triage a product question | The question and product metadata | choice() for intent (fit or sizing, compatibility, availability, shipping); boolean() for whether it's a support issue | Answer from product data, or hand off to support |
| Classify a search query | The raw query and recent catalog terms | choice() for intent (browse a category, find a specific product, check an order); score() for how specific the query is | Pick the results page layout, or redirect to order lookup |
| Categorize a new catalog item | Supplier title, description, and attributes | choice() over your leaf categories, or one call per level that passes the previous selection in state | Place the product, or hold it for a merchandiser when confidence is low |
| Flag a risky order note | Order note and shipping details | boolean() for a delivery instruction that conflicts with the address; boolean() for a request to alter the invoice | Pause fulfillment for review |
| Score a support chat for handoff | The conversation so far | score() for customer frustration; boolean() for a cancellation request | Escalate to a person before the customer asks |
Your application uses Jev's classifications and scores alongside inventory, account history, and business rules to decide whether to approve a return, change an order, or escalate a request. Explore seven practical Jev use cases for patterns outside commerce.
Copy link to headingBest practices
Copy link to headingAsk atomic questions and combine them in code
Jev works best when each question asks one well-scoped thing that a knowledgeable person could answer in a few seconds. If a question requires extended reasoning or weighs several independent factors, split it into one question per factor and combine the answers with your own logic. Each question is evaluated independently, so adding one doesn't change the answers to the others.
| Instead of | Ask | Then |
|---|---|---|
| "Rate this pull request" | 3 score() questions: test coverage, documentation, description clarity | Weight the three scores by importance in code |
| "Is this ticket a priority?" | A boolean() for urgency, a score() for business impact, a choice() for customer tier | Compute ticket priority from all three answers |
| "Should the agent run this command?" | A boolean() for destructive intent, a boolean() for touching production, a choice() for category | Require confirmation when any risk flag exceeds your threshold |
Copy link to headingDescribe options and levels instead of labeling them
choice() options are a map of keys to descriptions, and score() levels are ordered descriptions from lowest to highest. Writing 'Mostly negative with some positives' gives the model far more to match against than 'negative'. When an option key needs no description, pass null.
Copy link to headingKeep state focused
Input tokens are the only thing you pay for, so pass the fields the decision depends on rather than an entire record. For a review, that's the product name, title, body, and rating, not the customer's account or order history. state accepts an object, so you can select fields without serializing them yourself.
Copy link to headingSet thresholds per action, not per model
Read-only actions like showing a screen can tolerate a wrong guess, so a probability of 0.7 might be enough. Destructive actions need a higher bar, closer to 0.9 or above. Set the numbers from reviews you've already moderated, weighing that publishing spam costs more than holding a good review, and send anything under the threshold to a moderator rather than rejecting it.
Keep classification separate from authorization. Jev can tell you that a review looks promotional or that a command looks destructive. Whether to publish the review or run the command depends on rules Jev doesn't see, such as account status, policy, and permissions, and those checks apply whatever the probability.
Copy link to headingCreate the adapter once
Construct vercelGatewayDecider('typesafe-ai/jev') at module scope rather than inside a request handler. The adapter holds no per-request state, and reusing it avoids repeating the environment lookup on every call.
Copy link to headingTroubleshooting
Copy link to headingAuthentication errors (401 or 403)
Neither AI_GATEWAY_API_KEY nor VERCEL_OIDC_TOKEN is set, or the OIDC token has expired. Set an API key, or run vercel env pull .env.local to refresh the token, using vercel link first if the directory isn't linked to a project. If a 403 persists, check your access to the linked project and AI Gateway.
Copy link to headingdecide() throws before any request is made
Make sure questions contains at least one entry. Add a question if the map is empty, and rename any entry using the reserved key meta.
Copy link to headingEvaluation is unavailable through an OpenAI-compatible client
For Jev evaluation in TanStack AI, use vercelGatewayDecider with decide(). The vercelGatewayText adapter only handles text generation.
AI Gateway also supports evaluation through AI SDK, the native HTTP API, and the TypeSafe-compatible API. Its OpenAI-, Anthropic-, and Cohere-compatible endpoints don't support evaluation.
Copy link to headingThe request never completes
Evaluation runs on the server and needs a credential, so a decide() call from browser code fails. Wrap the call in a server route and call it over fetch, as in step four. For long-running requests, pass an abortSignal so your handler fails fast instead of holding the connection open.
Copy link to headingAnswers seem overconfident on your data
Run labeled examples through the same questions and compare predicted probabilities with observed outcomes. Inspect where errors cluster, such as reviews about damaged packaging landing in productQuality instead of shipping, revise unclear option and level descriptions, and evaluate the updated questions before choosing new thresholds.
Copy link to headingNext steps
- Read How to classify, route, and score with Jev and AI SDK for the same patterns with
experimental_evaluate, including how Jev differs from a language model - Follow Using TanStack AI with Vercel AI Gateway to stream chat, generate embeddings, and configure provider routing with the same adapter package
- See the TanStack AI Vercel AI Gateway adapter reference for
createVercelGatewayDeciderand Gateway routing options - Read What is Jev, TypeSafe AI's System One model? and When should you use Jev? to find other decisions in your application that fit a typed question
- Evaluate Jev's probabilities and choose a threshold using labeled examples from your own workflow
- See the AI Gateway evaluation docs and the Jev model page for pricing, limits, and the HTTP API
- Build a Jev agent with TanStack Start and TanStack AI