Imagine an upstream API starts returning 503s, and every client computes the same retry schedule. The moment it starts coming back, all of them retry at once and knock it over again before it finishes recovering.

Exponential backoff is the standard defense, though it only spreads those retries across a wider window. The total number of requests hitting the upstream stays the same.

This guide covers the formula, where jitter helps, the control that bounds retry volume, and which Vercel primitives retry on your behalf.

Key takeaways:

- Backoff spaces retries further apart, but the same number of requests still arrive at the failing service, so the fix for a retry storm is a limit on how much of your traffic can be retried.

- Clients that fail at the same moment wait the same amount of time and retry together, so each delay needs a random component to break up the pattern.

- Retrying an expired API key or a malformed request wastes every attempt, so check whether waiting can resolve the failure before scheduling a retry.

- Retry sits at a different layer for each Vercel primitive, so check whether the one you're using already retries before adding your own loop.

- Circuit breakers need more failure data than a short-lived function instance ever collects, which is why [AI Gateway](https://vercel.com/docs/ai-gateway/models-and-providers/model-fallbacks) makes those calls using data from across the fleet.

## [Copy link to heading](#what-is-exponential-backoff)What is exponential backoff?

Exponential backoff is a retry strategy that waits longer after each failed attempt, usually doubling the delay every time. You reach for it when a request fails for a reason that might clear on its own, such as a 429 rate limit, a 503 service unavailable, or a network timeout, because retrying immediately piles more load onto a service that is already struggling.

Each longer wait gives the upstream more room to recover, and the strategy stops being useful the moment a failure becomes permanent, since no amount of waiting can fix a 401 unauthorized or a 400 bad request.

### [Copy link to heading](#how-exponential-backoff-works-with-jitter)How exponential backoff works with jitter

The formula alone has a defect that appears only under correlated failure. Every client that failed at the same instant computes an identical schedule, so they retry together at 1 second, then together at 2 seconds, and the wave gets wider rather than smaller.

Randomizing each delay, called jitter, is what breaks that synchronization. Google Cloud's [June 2025 outage](https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1SsW) shows the cost. Service Control tasks restarted without randomized exponential backoff and created a herd effect on the Spanner table underneath, and recovery in us-central1 took around 2 hours and 40 minutes against roughly 40 minutes elsewhere, because engineers had to throttle task creation to keep the underlying infrastructure alive.

Four [jitter variants](https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter) dominate production use:

| Strategy | Formula | What it optimizes |
| --- | --- | --- |
| No jitter | `min(cap, base * 2^n)` | Nothing. It performs the most client work and takes the longest to complete |
| Full Jitter | `random(0, min(cap, base * 2^n))` | Lowest client work of the four, at slightly longer completion than Decorrelated Jitter |
| Equal Jitter | `temp/2 + random(0, temp/2)`, where `temp = min(cap, base * 2^n)` | A guaranteed minimum wait, at slightly more work than Full Jitter and much longer completion |
| Decorrelated Jitter | `min(cap, random(base, prev * 3))` | Fastest completion, at a higher call count than either Full or Equal Jitter |

Decorrelated Jitter has a failure mode worth knowing about. Once `prev * 3` consistently exceeds the cap, the output pins to the cap and stops randomizing, which returns clients to the synchronized schedule the jitter existed to prevent.

Jitter only changes when each retry arrives. The same number of requests still reaches the upstream, and on serverless, each of those requests runs inside a function with a duration limit of its own.

### [Copy link to heading](#benefits-of-exponential-backoff-for-serverless-teams)Benefits of exponential backoff for serverless teams

Backoff decides whether a struggling upstream recovers. On serverless, it also decides whether your own function survives the schedule it is running.

A growing delay affects five things at the same time:

- Recovery time for the upstream: Each doubled wait gives the failing service a longer uninterrupted stretch to drain queues, restart tasks, or add capacity. That recovery is what ends the incident, not the retries arriving during it.

- Protection against a synchronized wave: Randomized delays scatter clients that all failed at the same moment across a window of time. Without that, a service comes back up and gets knocked down again by everyone who was waiting for it.

- Room inside the function's time limit: A function on [Fluid compute](https://vercel.com/docs/fluid-compute) runs for 300 seconds by default, and up to 800 seconds on Pro and Enterprise. A retry schedule whose total wait passes that limit ends the invocation with `FUNCTION_INVOCATION_TIMEOUT`.

- Cost of the time spent waiting: Active CPU billing runs while your code executes and pauses while the function waits on I/O, such as model calls, database queries, and third-party APIs. A long wait on an upstream response costs less than the wall-clock number suggests.

- Safety of running the work twice: A retry is only safe when the same operation can run again without changing the result. GET, PUT, HEAD, DELETE, and OPTIONS are [idempotent methods](https://httpwg.org/specs/rfc9110.html#idempotent.methods), while POST and PATCH need an idempotency key before they are safe to repeat.

Function duration places a separate bound on the retry schedule. Waits of 1, 2, 4, 8, 16, and 30 seconds consume 61 seconds of sleep before the seventh attempt, which fits comfortably under a 300-second default and not at all under a `maxDuration` that someone dropped to 10 seconds for cost reasons.

Once retries need minutes rather than seconds, the function is the wrong place for them, and the retry loop becomes a set of components chosen deliberately.

## [Copy link to heading](#core-components-of-an-exponential-backoff-retry-policy)Core components of an exponential backoff retry policy

A retry policy is five decisions, and most incident reports trace back to one of them being left at a default. Each is worth setting explicitly rather than inheriting.

### [Copy link to heading](#base-delay,-multiplier,-and-cap)Base delay, multiplier, and cap

Three numbers define the schedule. The base delay is how long you wait after the first failure, the multiplier is what that wait gets multiplied by each time after, and the cap is the longest wait you allow. Vercel's [Workflow error handling course](https://vercel.com/academy/svelte-on-vercel/workflow-error-handling) uses `Math.min(1000 * 2^(attempt-1), 30000)`, which is a 1-second base, a multiplier of two, and a 30-second cap, producing waits of 1, 2, 4, 8, and 16 seconds.

The cap is what keeps the schedule usable. Without it, doubling reaches waits no one will sit through and no function will survive, so every attempt past the cap waits that maximum instead. The base is the number worth setting deliberately.

A database that fails over in under a second and a model provider shedding load for a minute need different starting waits, so you set it against the upstream's real recovery time rather than copying 1,000 milliseconds out of habit.

### [Copy link to heading](#jitter)Jitter

Jitter is randomization applied to the computed delay so that clients failing at the same moment do not retry at the same moment. It costs one line of code and is the highest-value addition to a naive backoff loop, and it belongs at every layer that computes a delay, including reconnect loops.

Vercel's [WebSocket documentation](https://vercel.com/docs/functions/websockets) shows the reconnect pattern as `reconnectDelay = Math.min(reconnectDelay * 2, 30000)`, which doubles correctly and does not randomize.

A WebSocket served by a [Vercel Function](https://vercel.com/docs/functions/limitations) closes when the function hits its maximum duration, so connected clients get disconnected on roughly the same cadence, which is precisely the condition that produces a synchronized reconnect wave. The cost of adding randomness is that individual completion times become unpredictable, and Full Jitter can produce a near-zero wait on any given attempt.

### [Copy link to heading](#error-classification)Error classification

Error classification decides whether a failure consumes a retry attempt at all. Retrying a 401 is wasted compute in the best case and an account lockout in the worst.

The split follows one question, which is whether waiting can change the outcome:

| Failure | Classification | Reason |
| --- | --- | --- |
| 429 rate limit | Retryable | Capacity returns on a schedule, the upstream controls |
| 503 service unavailable | Retryable | The upstream is expected to come back |
| Network timeout | Retryable | Transient path or load condition |
| 401 unauthorized | Fatal | Credentials do not become valid by waiting |
| 400 bad request | Fatal | The request is malformed on every attempt |
| Invalid input | Fatal | Validation produces the same result each time |

[Vercel Workflows](https://vercel.com/docs/workflows) makes this decision explicit through two error classes, and a hand-rolled loop should default to failing rather than retrying, so unclassified errors surface instead of quietly burning attempts.

### [Copy link to heading](#attempt-limits)Attempt limits

An attempt limit caps how many times a single logical request can be retried. Google's Site Reliability Engineering (SRE) practice sets a [per-request budget](https://sre.google/sre-book/handling-overload) of three attempts, on the reasoning that a request that has already landed on overloaded tasks three times is unlikely to find a healthy one on the fourth.

Ecosystem defaults cluster in the same range. The [AI SDK](https://ai-sdk.dev/docs/ai-sdk-core/settings), Vercel's software development kit (SDK) for building AI applications, ships `maxRetries: 2`, for three total attempts, and Vercel Workflows retries a failed step three times by default, for four total attempts.

This is also the limit that matters least, because three attempts per request bounds one user's experience and does nothing about aggregate load. A hundred thousand clients each stopping politely at three attempts still produce three hundred thousand requests against a service that cannot serve one.

### [Copy link to heading](#retry-budget)Retry budget

A retry budget caps the fraction of outgoing requests allowed to be retried. Each client tracks the ratio of retries to total requests across a recent window and refuses to retry once that ratio crosses the threshold, failing locally without touching the network.

The growth numbers are why this one is worth building. In Google's worst case, a datacenter rejecting a large portion of requests sees traffic grow to almost three times the original rate under a per-request limit of three attempts, and layering a 10% retry budget on top reduces that growth to roughly 1.1x.

A budget is a one-time addition to application code, and it is the only component here that bounds retry volume rather than retry timing. Everything else on this list makes the same number of retries arrive more politely.

## [Copy link to heading](#how-vercel-handles-exponential-backoff-across-its-retry-primitives)How Vercel handles exponential backoff across its retry primitives

Retry behavior on Vercel is set per primitive, so implementing a retry loop at a layer that already retries is the most common way teams create compounding by accident. This map is worth checking before writing any retry code:

| Primitive | Built-in retry | What your code owns |
| --- | --- | --- |
| Vercel Workflows | 3 retries by default, for 4 total attempts | Classifying each error as retryable or fatal |
| [Vercel Queues](https://vercel.com/docs/queues/concepts) | Configured delay for the first 32 attempts, forced exponential backoff after | Deduplicating messages and acknowledging poisoned ones |
| [Incremental Static Regeneration (ISR)](https://vercel.com/docs/incremental-static-regeneration) | Automatic 30-second retry time to live (TTL) on failure | Nothing. Stale content is served in the gap |
| Vercel Functions | None | All retry logic, inside the duration ceiling |
| [Cron jobs](https://vercel.com/docs/cron-jobs/manage-cron-jobs) | None | Everything. Failed invocations are never retried |
| AI SDK | `maxRetries: 2` by default | Overriding the default per call |
| AI Gateway | Automatic provider routing and fallback | Nothing for same-model failover. A cross-model fallback list is optional |

The row that catches teams out is cron jobs, where a failed invocation is lost until the next scheduled run and nothing in the platform brings it back.

### [Copy link to heading](#vercel-workflows-makes-the-retryable-or-fatal-decision-explicit)Vercel Workflows makes the retryable-or-fatal decision explicit

A hand-rolled retry loop buries its classification logic inside a catch block, where it drifts out of sync with the upstream's real error semantics and nobody notices until an auth failure quietly consumes four attempts. Vercel Workflows moves that decision into the type system, where throwing a `FatalError` skips retry logic entirely and throwing a `RetryableError` schedules another attempt after a delay you specify.

A step that classifies before backing off looks like this:

```
import { FatalError, RetryableError, getStepMetadata } from 'workflow';

export async function callUpstream(payload: Payload) {
  'use step';

  const response = await fetch(endpoint, { method: 'PUT', body: payload });

  if (response.status === 401 || response.status === 400) {
    throw new FatalError(`Not retryable: ${response.status}`);
  }

  if (!response.ok) {
    const { attempt } = getStepMetadata();
    throw new RetryableError(`Upstream returned ${response.status}`, {
      retryAfter: Math.min(1000 * 2 ** (attempt - 1), 30000),
    });
  }

  return response.json();
}
```

The `retryAfter` option accepts a duration string such as `"5m"`, a millisecond count, or a `Date`, so an upstream's own rate-limit signal can drive the wait instead of a formula. `getStepMetadata()` exposes the attempt number, which makes attempt-aware backoff possible without tracking state yourself. Workflows can also pause, resume, and hold state for minutes to months without a run-level duration limit, so a retry schedule that needs to outlive one function invocation has somewhere to live.

### [Copy link to heading](#vercel-queues-forces-backoff-and-expects-idempotent-consumers)Vercel Queues forces backoff and expects idempotent consumers

Queue consumers get written against the happy path, and the first duplicate delivery corrupts something that was never designed to run twice. Vercel Queues delivers at least once, with redelivery following ordinary causes such as a consumer crash, a deployment rollout, or a function timing out before it acknowledges. Queues respect your configured retry delay for the first 32 delivery attempts, then force exponential backoff to prevent runaway deliveries.

There is no [built-in dead-letter queue](https://vercel.com/i/dead-letter-queue), so poisoned messages get handled at the application level through the SDK's retry callback, and because messages with no prior delivery attempts are always prioritized over retried ones, a failing message falls to lower priority on its own instead of blocking the consumer. Underneath all of this, your handler has to survive running twice, which runs deeper than backoff and is covered in our [API idempotency guide](https://vercel.com/i/what-is-idempotency). The message lifetime is the other constraint to plan around, where the default TTL is 24 hours, and the maximum is 7 days.

### [Copy link to heading](#ai-gateway-moves-retry-and-fallback-out-of-application-code)AI Gateway moves retry and fallback out of application code

Multi-provider AI applications accumulate retry logic, fallback routing, and provider health checks in application code, and every one of those is an estimation problem that individual function instances are badly positioned to solve. AI Gateway handles all three at the routing layer, retrying the same model on a different provider when one degrades, with no fallback array configured in advance.

[Zo Computer](https://vercel.com/customers/how-zo-computer-improved-ai-reliability-20x-on-vercel) ran both paths simultaneously during rollout, producing a live A/B comparison under identical production conditions. In the after-switch period, the non-Vercel route recorded a 17.07% retry rate and a 10.38% POST error rate, compared with 0.34% and 0.45%, respectively, on Vercel. The average number of attempts per Vercel-routed chat was 1.00. The case study’s 20x improvement compares the 0.34% Vercel retry rate with Zo’s 7.52% pre-switch baseline.

[Cline](https://vercel.com/i/vercel-ai-gateway-vs-cloudflare-ai-gateway), the open-source coding agent, saw the same shape when it A/B tested its previous router against AI Gateway on live production traffic, with P99 streaming latency improving 10 to 14% and API error rates dropping 43.8%.

### [Copy link to heading](#fluid-compute-sets-the-wall-clock-budget-every-retry-schedule-runs-inside)Fluid compute sets the wall-clock budget every retry schedule runs inside

A retry schedule gets designed against an upstream's recovery time and then deployed into a function whose ceiling was configured for unrelated reasons. Vercel Functions have no built-in retry, so the whole schedule runs inside one invocation. The Fluid compute default is 300 seconds on all plans, with a maximum of 800 seconds on Pro and Enterprise. Supported Node.js and Python runtimes can opt into an extended 1,800-second maximum in beta. Active CPU billing pauses while the function waits on documented I/O, which lowers the cost of waiting without moving the wall-clock ceiling.

When a retry schedule needs the headroom, configure [maximum duration](https://vercel.com/docs/functions/configuring-functions/duration) on the specific function rather than as a project default:

vercel.json

```
{
  "$schema": "https://openapi.vercel.sh/vercel.json",
  "functions": {
    "app/api/sync/**/*": { "maxDuration": 800 }
  }
}
```

Anything that needs to keep retrying past that ceiling belongs in Vercel Workflows rather than in a longer function.

## [Copy link to heading](#five-best-practices-for-implementing-exponential-backoff-and-jitter)Five best practices for implementing exponential backoff and jitter

Those five components are settings. Turning them into a policy that holds during a real incident takes a handful of decisions about where the retry lives and what stops it.

### [Copy link to heading](#classify-errors-before-you-calculate-a-delay)Classify errors before you calculate a delay

Loops that treat every exception alike spend their full attempt budget on an expired API key and then report a timeout. The failure reads as a slow upstream in the logs, and finding the real cause costs an hour.

You want that check running ahead of the sleep rather than inside the catch block after it. Retryable failures get the backoff schedule, fatal failures throw immediately, and anything you have not classified fails fast so it reaches you as an error rather than as latency.

### [Copy link to heading](#honor-upstream-retry-signals-over-your-own-formula)Honor upstream retry signals over your own formula

The doubling schedule is a guess about when capacity returns. Some upstreams say so directly, and overriding that signal either retries too early and burns an attempt or waits far longer than needed.

Vercel's own 429 responses report the rate the caller is ramping toward, which means a correctly timed first retry succeeds. The [`@vercel/blob`](https://vercel.com/docs/vercel-blob/examples) [SDK](https://vercel.com/docs/vercel-blob/examples) follows the same pattern, where a `BlobServiceRateLimited` error carrying `error.retryAfter` replaces the computed delay outright.

### [Copy link to heading](#retry-at-the-layer-immediately-above-the-failure,-and-nowhere-else)Retry at the layer immediately above the failure, and nowhere else

Retries implemented at every layer compound multiplicatively rather than additively. Four attempts at each of three layers produce 4³, or 64 requests reaching the database for a single user action, which converts a small error rate into an outage.

Google's SRE guidance is to retry only at the layer immediately above the one rejecting the request, and to pass an explicit do-not-retry error upward once a request is exhausted. Before you tune any of those layers, work out which of them already retry, because that audit usually finds more than the tuning does.

### [Copy link to heading](#cap-the-retry-fraction,-not-only-the-retry-count)Cap the retry fraction, not only the retry count

Retry volume has a threshold past which a system enters a [metastable failure state](https://sigops.org/s/conferences/hotos/2021/papers/hotos21-s11-bronson.pdf) that it cannot leave on its own. The numbers make it concrete. An application running at 280 queries per second against a database that handles 300 hits a 10-second network interruption, the offered load doubles to 560 on recovery, goodput falls to zero, and it stays there after the interruption clears. Getting out requires dropping the load below 150 queries per second.

No backoff schedule moves any of those numbers, because the offered load is identical; it gets spaced out. A budget moves them by refusing to send, so some of your requests fail locally and never reach the network. Accepting those local failures is the point rather than a side effect.

### [Copy link to heading](#prefer-a-retry-budget-to-a-circuit-breaker-on-short-lived-instances)Prefer a retry budget to a circuit breaker on short-lived instances

A circuit breaker works by accumulating enough failure observations inside one client instance to estimate the true failure rate. Vercel distributes traffic across short-lived function instances that each see low individual volume, so per-instance estimates are noisy and the breaker either trips early or never trips.

Google's SRE book flags the same constraint for client-side throttling, noting that clients sending requests only sporadically have a drastically reduced view of backend state. A ratio-based budget degrades more gracefully under that constraint, and moving the decision into a routing layer removes the estimation problem entirely.

## [Copy link to heading](#ship-retry-logic-that-survives-a-bad-upstream-day-on-vercel)Ship retry logic that survives a bad upstream day on Vercel

A `maxRetries: 2` default bounds a single call. Whether a struggling service recovers is decided elsewhere, by the fraction of traffic allowed to be retried, the classification that keeps permanent failures from consuming attempts, and the layer that owns the retry in the first place. How much of that you write yourself depends on which primitives you build on.

Here's how Vercel handles retry behavior across the primitives engineering teams already run:

- Vercel Workflows: The type system enforces error classification rather than convention, so a permanent failure fails fast and a retryable one waits as long as the upstream asked it to.

- AI Gateway: Provider failover happens on failure of data from the whole fleet, which signals that no single function instance is positioned to collect.

- Vercel Queues: Delivery is at least once with backoff enforced by the platform, so a consumer that crashes mid-message gets the work again instead of dropping it.

- Fluid compute on Vercel Functions: Waiting on an upstream stops billing Active CPU, which changes what a long retry schedule costs to run.

- ISR: A failed revalidation keeps serving stale content instead of an error, so a broken upstream never becomes a broken page.

[Start a new project](https://vercel.com/new) to put these retry patterns into practice, or browse [Vercel templates](https://vercel.com/templates) for examples already wired up with Workflows and AI Gateway.

## [Copy link to heading](#frequently-asked-questions-about-exponential-backoff)Frequently asked questions about exponential backoff

### [Copy link to heading](#what-is-the-formula-for-exponential-backoff)What is the formula for exponential backoff?

The canonical formula is `Math.min(1000 * 2**(attempt-1), 30_000)`, which produces waits of 1, 2, 4, 8, and 16 seconds and then caps at 30 seconds. The cap prevents unbounded doubling from producing waits; no function invocation can survive.

### [Copy link to heading](#what-is-the-difference-between-full-jitter-and-decorrelated-jitter)What is the difference between Full Jitter and Decorrelated Jitter?

Full Jitter picks a random delay between zero and the capped exponential value, producing the lowest client work of the four common variants. Decorrelated Jitter completes faster but issues more calls, and it stops randomizing once its computed interval repeatedly exceeds the cap.

### [Copy link to heading](#do-vercel-functions-retry-failed-requests-automatically)Do Vercel Functions retry failed requests automatically?

No. Vercel Functions and cron jobs have no automatic retry, so the application code owns the logic. The schedule runs inside the function's configured wall-clock duration, which defaults to 300 seconds and maxes out at 800 seconds on Pro and Enterprise.

### [Copy link to heading](#how-many-times-does-vercel-workflows-retry-a-failed-step)How many times does Vercel Workflows retry a failed step?

Vercel Workflows retries a failed step 3 times by default, for a total of 4 attempts. Throwing a `FatalError` skips retries entirely, while a `RetryableError` retries after the delay has passed through its `retryAfter` option.

### [Copy link to heading](#does-the-ai-sdk-retry-failed-model-calls-by-default)Does the AI SDK retry failed model calls by default?

Yes. `maxRetries: 2` is the default on core functions, including `generateText`, `streamText`, and `embedMany`, for three total attempts. Set `maxRetries: 0` to disable retries, which is useful when a gateway or workflow layer already handles them.