---
title: "Handle Uncertainty"
description: "Predict the confidence boundary, distinguish confidence from choice probability, and extend an existing mocked test to cover every contact sample."
canonical_url: "https://vercel.com/academy/make-decisions-with-jev/follow-the-decision"
md_url: "https://vercel.com/academy/make-decisions-with-jev/follow-the-decision.md"
docset_id: "vercel-academy"
doc_version: "1.0"
last_updated: "2026-09-23T22:41:10.535Z"
content_type: "lesson"
course: "make-decisions-with-jev"
course_title: "Make Decisions with Jev"
prerequisites:  []
---

<agent-instructions>
Vercel Academy — structured learning, not reference docs.
Lessons are sequenced.
Adapt commands to the human's actual environment (OS, package manager, shell, editor) — detect from project context or ask, don't assume.
The lesson shows one path; if the human's project diverges, adapt concepts to their setup.
Preserve the learning goal over literal steps.
Quizzes are pedagogical — engage, don't spoil.
Quiz answers are included for your reference.
</agent-instructions>

# Handle Uncertainty

# Which answer does the app use?

The result can say **Selected by Jev** or **Selected by Luna**. Jev gets the first attempt. If its confidence is too low, the app asks Luna to make the same decision.

Open `lib/router.ts`. This branch decides whether to use Jev's answer:

```ts title="lib/router.ts: existing acceptance branch"
if (confidence !== null && confidence >= CONFIDENCE_THRESHOLD) {
  return { ...decision, destination, model: "typesafe-ai/jev" };
}
```

`CONFIDENCE_THRESHOLD` is `0.95`, or 95%. It's the minimum confidence this app accepts. When Jev returns valid confidence at or above that value, its selected team becomes the result. Otherwise, the code continues to the fallback call.

## Outcome

Predict the fallback behavior from raw confidence values and verify that both models receive the same routing policy.

## Hands-on exercise

### Read the two numbers

Jev assigns a probability to each possible destination. That collection of numbers is its **probability distribution**. The result's **Jev choice probability** displays the entry for the team Jev selected, taken from `answer.probabilities`.

**Jev confidence** summarizes how strongly that distribution favors one answer over the others. It arrives in `providerMetadata`, extra information returned by the model provider. In `lib/router.ts`, the app reads `providerMetadata.typesafe.confidence.destination`. It uses Zod, a validation library, to check that the value is a number between `0` and `1`.

The branch compares **confidence** with `0.95`, using the raw value. This threshold controls when the app accepts Jev. We check whether that answer is correct by comparing it with the expected team.

### Predict which model decides

For each pair of test values, predict whether the code accepts Jev's answer or calls Luna. These are artificial inputs chosen to test the code's decision rule, including whether it uses the wrong number. Assume both models return a valid destination.

| Jev confidence | Probability of Jev's selected option | Which model supplies the final team? |
| -------------- | ------------------------------------ | ------------------------------------ |
| `0.95`         | `0.99`                               |                                      |
| `0.94`         | `0.999`                              |                                      |
| `0.96`         | `0.70`                               |                                      |
| Missing        | `0.99`                               |                                      |

### Follow the second call

When Jev's confidence is below `0.95`, missing, or invalid, execution continues to `generateText`. A Jev error also reaches that call. Find this line in its options:

```ts title="lib/router.ts: existing fallback input"
prompt: JSON.stringify({ questions, state }),
```

The **prompt** is the input sent to Luna. `JSON.stringify` converts our question and form fields into text for that input. Luna receives the same information as Jev, without Jev's answer, and makes its own choice. Once the code checks that choice against the allowed destinations, Luna's answer becomes final.

This is the structured-output approach we discussed in Meet Jev. Find `Output.object` in the same call: it defines the shape Luna must return, with a `destination` chosen from the allowed IDs. Both approaches can choose a team. Jev's `evaluate` call also supplies the choice probabilities and confidence we inspect in the result.

In the UI, **Selected by** identifies the model that supplied the final destination. Jev's statistics remain attached to the result even if Luna chooses another team. If both models fail, the routing function rejects and the form reports a failure.

### Check the policy with controlled answers

Open `lib/router.test.ts`. The helper `mockJev(confidence, probability)` supplies fixed model answers so we can force a particular branch. Find the cases for `0.95`, `0.94`, and **does not substitute selected probability for confidence**. The `expect(...)` lines are **assertions**: checks that fail the test when the result differs from what we specified. Compare them with your predictions.

Run this test file from the app folder in your second terminal:

```sh
pnpm exec vitest run lib/router.test.ts
```

`pnpm exec` runs a tool installed in this project. Here it runs **Vitest**, the test runner, for `lib/router.test.ts`. Check that the terminal reports the tests as passed.

The suite also checks `0.94999`: the UI rounds it to `95.00%`, but the raw value is below the threshold and calls Luna.

### Include our new request

Find the test whose name begins **passes identical state and criteria**. It currently uses the first sample, so it doesn't exercise the cancellation sample we added.

Change it to `it.each(example.samples)`. This runs the same test once for every sample in the array, passing the current entry to the callback as `sample`. Use `sample.values` in the call to `routeSubmission`.

Keep the assertions that compare the two models' inputs and exclude Jev's statistics from Luna's prompt. The **Solution** section provides the full replacement.

The fake Luna model always returns `contact_triage` in this test. That lets us check whether the router uses the fallback's answer, including when it differs from Jev's choice.

## Try It

Run the focused suite again and check that the input-preservation test passes for each of the four samples. Inspect **fails when both providers fail** too: its assertion expects the routing function to reject.

Then explain this possible result without running a model: Jev chooses refunds with confidence `0.80`, and Luna chooses account access. Which team is final, and whose confidence will the UI show? Check your explanation against the solution.

### If something goes wrong

**The new test still checks one request:** make sure the call uses `sample.values`. Changing the test heading alone leaves the old `submission` input in place.

**Your prediction follows the larger number:** locate the metadata read in `lib/router.ts`. `selectedProbability` is saved for inspection; the acceptance branch uses `confidence`.

## Checkpoint

Save `lib/router.test.ts` and your four predictions in `routing-notes.md`. Explain why a high selected-option probability does not change the confidence comparison.

## Done-When

- [ ] You can explain each prediction using the acceptance branch.
- [ ] The input-preservation test runs once per contact sample and passes.
- [ ] You can identify the final destination when the models disagree.
- [ ] You can locate the test for failure of both providers.

## Solution

The predictions, in order, are **Jev, Luna, Jev, Luna**. The third row accepts Jev because confidence exceeds the threshold, even though its selected-option probability is lower. Missing confidence sends the fourth row to Luna.

In the disagreement example, account access is final because Luna supplies it. The displayed `0.80` confidence still describes Jev's refund choice.

Replace the single input-preservation test with this block inside the existing `describe("routing policy", ...)`. It reuses the file's imports and mock helpers:

```ts title="lib/router.test.ts: all contact samples"
it.each(example.samples)(
  "passes identical state and criteria to the fallback for $label without Jev's answer",
  async (sample) => {
    const original = mockJev(0.8).model;
    const evaluateCall = vi.fn(original.doEvaluate);
    const jev = new Experimental_EvaluationMockModelV4({
      doEvaluate: evaluateCall,
    });
    const luna = mockLuna("contact_triage");
    const result = await routeSubmission(example, sample.values, {
      jev,
      luna,
    });
    const [[evaluation]] = evaluateCall.mock.calls;
    const prompt = JSON.stringify(luna.doGenerateCalls[0].prompt);
    expect(prompt).toContain(
      JSON.stringify(
        JSON.stringify({
          questions: evaluation.questions,
          state: evaluation.state,
        })
      ).slice(1, -1)
    );
    expect(prompt).not.toContain("probabilities");
    expect(prompt).not.toContain("confidence");
    expect(result.destination.id).toBe("contact_triage");
  }
);
```

`vi.fn` records calls to the fake Jev model. The test reads that record to see what Jev received, then checks for the same question and state in Luna's recorded input. The nested `JSON.stringify` calls handle the escaped JSON inside that recorded prompt.

The last assertion checks that the router returns Luna's chosen team. With `it.each`, we now check this path for every contact sample, including our cancellation request.


---

[Full course index](/academy/llms.txt) · [Sitemap](/academy/sitemap.md)