Timeout and Retry Strategy Design for Multi-Step Agent Loops

Multi-step agent loops don't fail the way normal software fails. A null pointer exception announces itself, points to a line number, and dies the same way every time you run it. An agent loop can succeed nine times on identical input and then, on the tenth run, quietly call the same tool a large number of times in under a minute with no outward sign of failure. The thesis here is straightforward: timeouts and retries in agent systems need to be designed as three separate failure domains, LLM calls, tool invocations, and the loop itself, each with its own bounds, because the failures that actually cost money are the ones that look like progress.

That's a different engineering problem than the one most teams think they're solving. A single LLM API call fails somewhere in the range of 1 to 5 percent of the time, from rate limits, timeouts, or server-side errors. On its own, that's a manageable number, the kind of thing a retry with backoff handles without much drama. The real threat is what happens when those individual failure rates compound across a chain of steps. At 99 percent reliability per step, a 20-step agent task only completes successfully about 82 percent of the time. That gap, the space between "each step almost always works" and "the whole task usually works," is the entire reason per-call error handling isn't enough on its own. You need something watching the loop as a whole, in addition to the calls inside it.

A taxonomy of how agent loops break, grounded in production data

The most rigorous public accounting of how these systems fail comes from the MAST taxonomy, built from more than 1,600 annotated traces across seven agent frameworks, with 150 traces analyzed in depth and an inter-annotator agreement of kappa = 0.88, a level of consistency that's rare for a problem this qualitative. The traces break into three categories. System design issues account for 44.2 percent of failures, the largest share and, not coincidentally, the category most addressable by better timeout and retry architecture. Inter-agent misalignment makes up 32.3 percent. Task verification gaps cover the remaining 23.5 percent.

Underneath those categories are specific failure modes, each with a distinct signature. Step Repetition appears in 15.7 percent of failures: the agent calls the same tool with the same arguments, over and over, with no internal mechanism to notice it's stuck. Disobey Task Specification accounts for 13.2 percent. Premature Termination, the mirror image of repetition, occurs in 9.1 percent of cases, where the loop quits before the task is actually finished. And Reasoning-Action Mismatch, at 8.2 percent, describes something subtler: the agent states one plan in its reasoning trace and then executes a tool call that does something different.

Root-cause analysis in the same body of work splits failures into specification (42 percent), coordination (37 percent), and verification (21 percent). Specification and coordination together account for close to 80 percent of what goes wrong. That's a meaningful number, because it means the dominant failure mode in agent systems is not the model being dumb. It's the surrounding system failing to specify boundaries or coordinate handoffs correctly. Model quality improvements won't fix a structural gap in how a loop is bounded.

Five production incidents and the specific costs of runaway loops

Numbers on a taxonomy chart are one thing. What they look like in production is another, and a handful of documented incidents make the shape of the problem concrete.

Take the OpenClaw idle-timeout loop. A model call hit an idle timeout. The upstream layer, doing what it was built to do, retried it. The same idle-timeout condition triggered again, and again, in a tight loop with no circuit breaker to interrupt it. Two separate documented events logged 761 and 1,384 model calls within 60 seconds each. The estimated cost landed around $30, modest in isolation, but the fix that shipped the next day tells you how seriously the team treated it: a circuit breaker capping consecutive idle-timeout model calls at 5 by default, with any successful output resetting the counter back to zero.

Claude Code had its own version of this, on a larger scale. A session with a heavily occupied context window kept sending repeated requests for hours after a compaction attempt, racking up more than $500 in unexpected usage before anyone caught it. The trigger was different from the OpenClaw case, a context-compaction failure rather than an idle timeout, but the underlying governance gap was identical: repeated model calls kept consuming tokens with nothing in place to cut them off.

A separate Claude Code incident shows a variant that's arguably more dangerous because it's quieter. A sub-agent consumed a huge quantity of tokens in an infinite loop that ran for 4.6 hours. No burst of hundreds of calls per minute, no obvious spike, just a slow, steady bleed that a per-agent step limit would never have caught, because the loop wasn't fast. It was patient.

Treating timeouts as three separate bounds, not one deadline

The instinct most teams start with is a single wall-clock timeout wrapped around the whole workflow. It's the obvious move, and it's also the wrong one, because a wall-clock deadline has no way to tell the difference between an agent working through something legitimately hard and an agent spinning in a loop it can't detect. Both look identical from the outside: time is passing, no result yet.

What actually works is three separate bounds, layered so each one covers the blind spot of the other two.

The wall-clock timeout is still necessary. It kills hung processes and runs that have gone on too long, independent of how many steps were taken or how many tokens were burned. Its blind spot can still cause problems: a fast-spinning loop can exhaust an entire token budget well before the clock runs out, and a legitimate long-running task can get killed just because it needed more time than the deadline allowed.

The step-count ceiling catches what the wall-clock misses, namely a loop calling the same tool repeatedly, fast enough that it never approaches the time limit. LangGraph ships with a recursion_limit that defaults to 25, which is a useful anchor point for calibration conversations even outside that specific framework. As a starting point, 25 steps is generous enough to accommodate genuine multi-hop reasoning while still catching a runaway loop before it turns expensive. One point of reference puts most well-designed agents finishing tasks in 5 to 15 steps, and that range is more useful for calibrating a ceiling on a specific workflow than treating 25 as some kind of universal cap. The step ceiling's own blind spot: an agent can stay under 25 steps and still burn an enormous amount of money if each individual step is dragging along a massive context window.

That's where the token budget comes in, set separately for input and output tokens. When a loop approaches its budget, the right behavior is graceful degradation, instructing the agent to wrap up with a partial result instead of attempting one more reasoning pass and getting cut off mid-thought. Reported figures on observability-driven token optimization point to waste reductions in complex agent loops, which suggests that a lot of token spend in production systems today is just unmonitored, not necessary.

None of these three bounds substitutes for the other two. That's the actual design principle: wall-clock, step count, and token budget, running together, each one closing a gap the others leave open.

An open-source agent harness on GitHub illustrates how this looks when calibrated per task. For a benchmark-style coding task, the bounds are set to a relatively tight iteration count, modest input and output token ceilings, and a short wall-clock limit. For SWE-bench tasks, which involve far more exploration of a codebase, the bounds widen substantially: substantially higher iteration counts, input and output token ceilings, and a much longer wall-clock allowance. The harness terminates with SIGTERM first, then SIGKILL if the process doesn't respond, a two-stage kill sequence that matters in practice because a hard kill mid-write can leave things in a worse state than a clean shutdown. These figures are a demonstration of how the three dimensions get tuned together against the actual complexity of a task, not universal defaults. They're a demonstration of how the three dimensions get tuned together against the actual complexity of a task.

Retry strategy design across the two levels where loops fail

Retry logic has a scope problem that's easy to miss until it's already expensive. If a workflow has 8 steps and each one independently retries up to 3 times, the worst case for a single run is 24 total API calls, and if any of those calls trigger a tool that itself makes API calls, the multiplication compounds again from there. Retry logic designed in isolation, one step at a time, has no way to see that compounding effect.

Two levels need separate handling. The first is per-call retry, sitting at the infrastructure layer, applied to individual LLM calls and tool invocations. The standard pattern here is exponential backoff with jitter, where the delay between attempts roughly doubles each time and a randomized jitter factor prevents a fleet of retrying clients from all hammering the server in the same instant. AWS research on distributed systems found this pattern reduces retry storms by 60 to 80 percent, a wide but meaningful range depending on how aggressively the jitter is tuned. Retry counts shouldn't be uniform across error types, either: a rate-limit error, a server-side error response, and a dropped network connection each warrant different retry behavior, and treating them identically wastes attempts on failures that won't resolve by retrying and gives up too early on ones that would. Tenacity, an Apache 2.0-licensed Python library, is a solid reference point for implementing this without writing backoff logic from scratch. Tooling gaps appear in this pattern too: StructuredTool instances created through MultiServerMCPClient in the LangChain ecosystem don't natively support retry configuration through with_retries, and the documented workaround is a proxy or wrapper pattern using the wrapt library.

The second level is the workflow-level retry budget, a shared pool of attempts across the entire run rather than a count reset at each step. If the budget is 5 total retries for the whole workflow and Step 1 consumes 2 of them, only 3 remain for everything downstream. That constraint is what actually prevents the worst-case compounding scenario, and it forces the workflow to fail explicitly once the budget runs out, rather than silently draining resources across steps with no single point where anyone would notice. Spending controls belong inside the execution path itself, enforced as the loop runs, not discovered after the fact in a billing invoice.

There's a retry failure mode that neither of these levels can see, because it happens inside the model's own reasoning rather than in the infrastructure. It happens inside the model's own reasoning. When a tool call times out or throws an error, the agent frequently doesn't retry the same call. It re-plans: that approach didn't work, so it tries something else. The trouble is that "something else" often turns out to be the same tool, called again with slightly different parameters, which is a retry in every functional sense except that no retry logic is tracking it. One account from an online forum thread on agent behavior put it this way: the agent saw "timeout" as "try a different approach" and kept running. Infrastructure-level circuit breakers have no visibility into this, because from their vantage point, each call looks like a fresh, valid request. Catching it requires detection at the loop level, watching the sequence of actions over time rather than judging any single call in isolation.

And some failures should not be retried. A tool returning a non-transient error, a task that requires a judgment call the agent has no authority to make, or a situation where getting the answer right is more important than getting it fast: these call for stopping, not for another attempt with slightly different parameters. Human escalation deserves to be treated as a first-class outcome of the retry system, not an afterthought bolted on when everything else fails. After a defined number of retries, the correct behavior is to hand the problem to a person, not to try again with more creativity.