AI Gateway adds confidence-based decision fallbacks

AI Gateway can now escalate a decision request to a fallback model when the primary model's answer trips a confidence condition you set.

Conditions can be combined, so an escalation can depend on more than one signal. The plain model names you already list in models keep catching outright errors, so existing fallbacks are unaffected. Confidence conditions cover Choice and Score questions. Boolean questions escalate on a probability range instead.

Set the fallback by adding one conditional object to providerOptions.gateway.models. Requests without it keep their existing behavior.

1
{
2
  "gateway": {
3
    "models": [
4
      {
5
        "model": "openai/gpt-6-astra",
6
        "when": {
7
          "question": "intent",
8
          "confidenceBelow": 0.6
9
        }
10
      }
11
    ]
12
  }
13
}

Leave out question and the condition checks every question of the matching type, so { "confidenceBelow": 0.6 } escalates when any Choice or Score answer falls below 0.6.

Because a triggered fallback runs a second decision, it bills both stages.

Decision fallbacks are in beta. Learn more in the decision fallbacks documentation.