Error Recovery and Retries in AI agents

Module 11 · AI Agents · 17 concepts

In depth / 15 lines

Error Recovery and Retries, in more detail

Every agent that touches the outside world will spend part of its life handling failure.

Networks time out, APIs rate limit, files are missing, and arguments are occasionally nonsense.

How those failures are surfaced to the model decides whether the agent recovers or spirals.

The first rule is to pass the real error text through rather than a generic failure.

A message saying the date field expects ISO format is directly actionable; failed is not.

Models correct their own tool calls reliably when told precisely what was wrong.

The second rule is to distinguish transient failures from permanent ones.

A timeout or a rate limit should be retried by the runtime, with exponential backoff and jitter.

That retry need not involve the model at all, and involving it wastes tokens.

A validation error or a missing resource will never succeed on repetition, so it must go to the model.

The third rule is that every loop needs a ceiling.

An agent that retries the same broken call indefinitely produces a large bill and no progress.

Limit attempts per tool and steps per run, and fail loudly when a limit is hit.

Idempotency matters once actions have side effects, because a retried payment is not free.

Log every failure with its arguments, since recurring tool errors usually mean a bad tool description.

Deep dive

What is underneath Error Recovery and Retries

Retry with exponential backoff

t_n = \min\big( t_{\max}, \; t_0 \cdot b^{\,n} \big)

$t_0$ : the base delay, often one second.
$b$ : the multiplier, conventionally 2.
$n$ : the attempt number.

Waiting a constant interval between retries synchronises every client into the same rhythm, which is what turns a brief overload into a sustained one.

t_n = \mathrm{rand}\big(0, \; \min(t_{\max}, \; t_0 b^{n})\big)

Full jitter spreads the retries across the whole window, and it is measurably better than backoff alone at clearing a queue.

1, \; 2, \; 4, \; 8, \; 16 \text{ seconds}

Five attempts covers about half a minute, which is long enough for a transient fault and short enough that a caller is still waiting.

Not every error should be retried

Retry on 429, 500, 502, 503, 504 and on timeouts.

Rate limits and server faults are transient by definition, and the same request will probably succeed later.

Do not retry 400, 401, 403, 404 or 422.

A malformed request, a bad key or a missing resource will fail identically every time, and retrying is a way to turn one error into five.

Some servers send a Retry-After header.

When the server states a delay, that number wins over any local formula, because it is the only party that knows when it will be ready.

The agent case is different

A tool error inside an agent loop is not only an operational failure. It is information the model can act on.

Catch the exception at the tool boundary rather than letting it kill the loop.
Return it as an observation, with the message and enough context to act on.
Let the model decide: fix the arguments, try another tool, or report that it cannot proceed.
Count consecutive failures of the same tool.
After a small number, stop offering that path and say so in the context.

Here an error is an observation, not an exception.

That reframing is what separates an agent that recovers from one that crashes, and it costs a try block.

Guarding against the repeat

a_t = a_{t-1} \implies \text{intervene}

A model that receives the same error twice often tries the same call a third time, since its context now contains two examples of that call being the thing to do.

The fix is to state it explicitly in the returned observation: this exact call has already failed twice, try something different.

The usual guard is a circuit breaker: it opens after $k$ failures and tries again after a pause.

For an external service, a breaker stops the calls entirely for a period, which protects both the caller's budget and the service's recovery.

Idempotency, or retries make it worse

Retrying a payment can charge the customer twice.

A retry after a timeout is dangerous whenever the first attempt may have succeeded before the response was lost.

The fix is an idempotency key: the same key always has the same effect.

Any tool with a side effect needs a key the server can deduplicate on, or the retry logic is a mechanism for doing things twice.

Read-only tools are safe to retry freely, which is a good reason to separate the two kinds in an agent's tool list rather than treating them alike.

Test yourself

One question on this card. Every option explains itself once you pick, right or wrong.

Question 1

A tool call fails with a validation error. What should the agent be given?

AI Agents

17 concepts in this module, then a graded test with every answer explained.

Study this module →