Which words change the decision?

“Cancel next month's renewal.” Our rule sends that to billing. Add “help me recover my account first,” and support has work to do before billing can help.

We'll keep the rest of the form steady and change what Alex asks us to resolve. Copy this table into the routing-notes.md we started in lesson 1.1:

routing-notes.md: revised policy

Date:
Subject for A–C: Cancel next renewal
Account context for A–C: Pro plan, workspace Daybreak
 
| Case | Expected team | Returned team | Selected by | Match? |
| --- | --- | --- | --- | --- |
| A | | | | |
| B | | | | |
| C | | | | |
| D | | | | |
| Fresh request 1 | | | | |
| Fresh request 2 | | | | |

Keep the submitted text with each row so we can compare the wording when an answer changes. If a request fails, record its error message. Keep the criteria unchanged while we run all six requests.

Outcome

Compare six requests and explain one result using the submitted fields, routing criteria, and deciding model.

Open your local /contact page and load Cancel next renewal. If you stopped the app, run pnpm dev from the project folder to start it again. Keep the sample's name, email, subject, and account context for A through C. Write all three expected teams before submitting.

Case	Message
A: Stop renewal	Please cancel next month's renewal. Keep my access until the current paid month ends. I am not asking for money back.
B: Return payment	Please cancel next month's renewal. I also want yesterday's renewal payment returned. Please handle the refund first.
C: Access first	Please cancel next month's renewal. First, help me recover my account. I lost my authentication device and cannot sign in.

Submit A and record the result before editing the message. Repeat for B and C. Identify the phrase that should send each request to a different team. If the returned destination stays the same, we have a specific difference to investigate.

For D, change the subject to Need a hand, clear account context, and use Please cancel it. Predict its team, then submit. A subject of Cancel next renewal would answer the question we're deliberately leaving open. The model receives the whole form.

Investigate one result

Choose a request whose result differed from your prediction. If they all matched, use B and explain why it belongs to Billing / Refunds.

If that request is no longer displayed, enter its exact fields again and submit. Record this as a separate run in your diagnosis; model answers can vary between submissions. Keep the original result in your table.

Start with the submitted fields. Does the subject add information missing from the message? Next, read Selected by and expand Decision details, then Raw result. This is the data behind the displayed answer.

destination.id is the final team ID. Inside jev, destination records Jev's original choice; confidence and selectedProbability describe that choice. Save those values in your notes. Write Unavailable for missing statistics or a null jev entry.

Also record fallbackReason. It tells us whether Luna was called because of low confidence, missing or invalid confidence, or a Jev error. A null fallbackReason means the app accepted Jev's answer.

Now compare the request with the criterion for the expected team and the criterion for the returned team. Write a diagnosis using the exact phrase that mattered:

routing-notes.md: diagnosis

Case:
Phrase that determines the requested resolution:
Relevant criterion:
Final deciding model:
Why I expected this team:
What I will check or change next:

If both teams' criteria could apply, note how you would clarify the rule. If the rule clearly identifies one team and the answer names another, save the request as a case to retest. Finish the six requests before editing the criteria so all six results use the same policy.

Check fresh wording

Leave the criteria fixed and use the subject Account request with account context Pro plan, workspace Daybreak. Predict and submit:

Please turn off auto-renew for our workspace. This month's payment is fine and we want to use the remaining time.
Our workspace was charged twice for the same month. Please send the duplicate payment back.

These messages express the same needs with different wording. Keep the criteria fixed while testing them. After another policy edit, write fresh requests to check whether the rule works beyond the examples used to develop it.

Try It

Count matches out of four for A through D, then out of two for the fresh requests. Keep any errors visible in those groups. Record the deciding model beside each answer so we can tell whether Jev or Luna supplied it.

Now apply any criterion change you proposed and rerun A through D. Keep the earlier results alongside the new ones to see whether the change helped or sent another request to the wrong team.

Finish the code checks in your second terminal:

pnpm validate

Check that each validation step passes. The mocked tests exercise the routing branches and form validation. Compare your live results with the expected teams to check how the models apply your criteria.

If something goes wrong

Every message goes to the same team: verify that you submitted after each edit. Compare the new result with the exact input, then check for overlapping criteria. Preserve the disagreement before trying a revision.

“Please cancel it” goes to a specific team: inspect the subject and account context. Leftover renewal information can make this request more specific than the message alone suggests.

Checkpoint

Save the six results and one diagnosis, including any extra run used to investigate it. If you edited a criterion after the comparison, save the change and the repeated A through D results alongside the originals.

Done-When

The observations include all six exact inputs and their expected and returned teams.
A through C keep the same form context; D removes the cancellation clues.
You can explain one result using the request, the relevant criterion, and the deciding model.
Local validation passes and model errors remain visible in your notes.

Solution

These labels follow the policy we wrote. Compare them with the live answers you recorded:

Case	Expected destination	Decisive evidence
A	`billing_invoices`	Stop a future renewal; no money back.
B	`billing_refunds`	“Please handle the refund first.”
C	`support_access`	“First, help me recover my account.”
D	`contact_triage`	The whole submission leaves “it” unidentified.
Fresh request 1	`billing_invoices`	Turn off auto-renew; keep the paid period.
Fresh request 2	`billing_refunds`	Return the duplicate payment.

B asks to return a collected payment and explicitly says “refund first,” so billing_refunds is the expected team. If the returned team is invoices, check that the invoice criterion excludes requests to return money and that the instructions preserve the refund rule. If those are present, record the disagreement as a case to retest. Keep the confidence threshold at 0.95 while comparing changes to the criteria.

Six requests give us examples to revisit when we change the policy. Write a seventh request that you find difficult to assign. Decide which team should handle it before asking the app.