Vercel Logo

Choose the model deliberately

The prototype calls a model through AI Gateway, but the choice is buried in one route and the response is unstructured text. Preparing it for production begins by making the provider, model, schema, data rules, and spending limit easy to review.

Bring in the implementation scaffold

Stay on course-work. The implementation-start tag contains the request UI, database plumbing, evaluation runner, and unfinished model, policy, and workflow functions. Bring those files into your branch:

git fetch upstream --tags
git checkout implementation-start -- .env.example .gitignore app evals lib next.config.ts package.json pnpm-lock.yaml scripts tests workflows
pnpm install
git add .
git commit -m "chore: add governed application scaffold"

This is application plumbing, not the answer key. pnpm test and pnpm eval:policy should fail until you implement the next lessons. Your original production deployment remains on main; previews from course-work show the new implementation.

Implement the assessment

Open lib/ai.ts. Name the model and the behavior version separately:

export const ASSESSMENT_MODEL = "openai/gpt-5.4-mini";
export const ASSESSMENT_VERSION = "vendor-assessment-v1";

Use a currently supported model from the AI Gateway model catalog; the identifier above is an example, not a permanent recommendation. Keep the version separate from the model. A prompt or schema change can alter behavior even when the provider and model remain unchanged.

Implement assessVendorRequest() with generateText() and Output.object(). The checked-in assessmentSchema allows a category, suggested risk, at most five missing-information items, and a short summary. Put the vendor request inside a labeled data block and tell the model that its contents are untrusted. The model may describe a request. It may not approve one.

Write down the Gateway rules

Write down the rules before changing dashboard settings:

AI task: identify missing information and suggest a risk classification
Data sent to the model:
Approved providers:
Approved or denied models:
Retention requirement:
Prompt-training requirement:
Production authentication method:
Local or external key owner:
Budget applies to (team, project, or API key), limit, and refresh period:
Failure behavior when a request is blocked:

AI Gateway provides one place to enforce these rules across supported frameworks and API formats. Use the current control that matches the requirement:

  • A team-wide provider allowlist restricts which providers may serve requests.
  • Routing rules can deny or rewrite requests for specific models. Routing rules are in beta.
  • Budgets can cap spend at the team, project, or API-key level. API-key budgets are useful when the application must use a key outside Vercel.
  • Team or request-level data controls enforce supported retention and prompt-training requirements.

Provider and model controls solve different problems. Record both instead of using “model allowlist” as a general label for every restriction.

Verify production authentication

For deployments on Vercel, use the platform’s automatically supplied OIDC identity. Keep AI_GATEWAY_API_KEY for local development, live evaluations, or runtimes where Vercel OIDC is unavailable. If both are configured, the API key takes precedence.

Confirm:

  • No key value appears in source, Git history, logs, or screenshots
  • Preview and Production use the intended Vercel project identity
  • Any local or external key has an owner, budget, and rotation procedure
  • The budget applies to the intended team, project, or API key
  • Authentication changes do not require changing application code

Do not create a hardcoded secret for the sake of removing it later.

Compare two models

Run the baseline requests recorded in Deploy the Prototype against two suitable models. Record:

  • Whether the output satisfies the expected shape
  • Differences in missing-information detection
  • Latency
  • Token use and cost
  • Any behavior that changes the application’s risk

The goal is not to crown a universal winner. Choose a model for this task and document why in docs/model-record.md.

Inspect usage

Open AI Gateway usage and locate requests from Vendor Review. Confirm that model, project or credential, request volume, token use, and cost can be traced to their source. A gateway is useful here because model choice, authentication, usage, and budgets live in one place without binding the application to one provider SDK.

Test one policy failure. Request a blocked provider or model in a safe environment, or lower a test key budget until a request is rejected. Confirm that Vendor Review shows a visible failure and does not turn the missing assessment into an approved business decision.

Commit the first implementation checkpoint:

git add lib/ai.ts docs/model-record.md
git commit -m "feat: add structured vendor assessment"

Summary

AI Gateway gives a company one place to control which models applications may use, how requests are routed, and where usage and spend are visible. Applications keep a consistent integration while the organization changes providers or policy.

Check your work

Run one baseline request and inspect the returned object. It should match the schema and contain no approval decision. Then cause one safe Gateway rejection. Vendor Review should show an assessment failure while preserving the request for later review.

Compare your implementation with lib/ai.ts on complete.

Was this helpful?

supported.