Self-Healing Tool Middleware: A Resilience Layer Between Agent and Tools

Overview#

Teams ship an agent that works in the demo, then watch it die in production on a single bad tool call. The model invents a tool name that does not exist, a flaky API times out once, the loop spins on the same call, or one oversized result blows the context window. Any one of those ends the run, and none of them are the model’s fault. They are the seam between the agent and its tools, left unguarded.

This is our resilience layer for that seam. It is a composable, model-agnostic middleware stack that interposes on every tool call before it reaches the model and recovers from the four failure modes that silently kill agent runs. The recovery is transparent: from the agent’s point of view, the tool simply succeeded. Because every signal it acts on is mechanical, an invented name, an error class, a repetition count, a payload size, the core carries no model SDK and runs no secondary LLM judge. It is deterministic, cheap, and stackable.

Figure 1 - The unguarded seam: The point where a tool name and arguments become a result is where production agents fail. The middleware sits on that seam, stacks four recovery layers in a defensible order, and returns a clean result up to the model. The agent never learns a call went wrong.

The Problem#

The reliability math is the whole motivation. A 10-step workflow at 90% per-step reliability completes end-to-end only about 35% of the time, because 0.9 to the tenth power is 0.35. Climbing the nines, from 90% to 99% to 99.9% per step, is what separates a demo from a production system, and you do not climb them by asking the model to try harder. You climb them by engineering the recovery at the tool seam.

The default failure is silent because each failure mode looks like a different problem. A hallucinated tool name reads as a model bug. A transient timeout reads as an infrastructure flake. A stuck loop reads as a prompting issue. An oversized payload reads as a context-window limit. Treated separately, each gets a one-off patch. Treated as one class, they are all the same shape: a mechanical fault at the tool call that the model should never have to see, let alone reason its way out of.

KEY INSIGHT: These four failures look unrelated in a postmortem, so they get four unrelated patches. They are one class: a mechanical fault at the tool seam that the model should never have to reason about. Handling them in one composable layer is what turns a brittle loop into a self-correcting one.

What We Built: The Four Layers#

Each layer is a wrapper around the tool-execution step, applied in order, and independently unit-tested. The order is deliberate: repair the name before you retry it, break a loop before you keep retrying inside it, offload the payload only once a call has actually succeeded.

Figure 3 - Four mechanical signals, four recovery moves: Each layer keys off a signal that needs no model to read. ToolNameRepair reads the invented name, RetryWithBackoff reads the error class, LoopBreaker reads the repetition count, ResultOffload reads the payload size. No layer makes a quality judgment, which is what keeps the whole stack deterministic and free of a secondary LLM.

Every threshold is a keyword argument, not a hard-coded constant. The name-repair score cutoff, the loop threshold and window, the retry attempts and backoff timing, and the offload byte threshold are all tunable per workload, because the right values depend on the client’s tools and SLA, not on ours.

KEY INSIGHT: The recovery is transparent to the model on purpose. Every signal is mechanical, so no layer needs to understand what the tool does. That is what lets the core ship without a model SDK, run deterministically, and cost nothing per recovery, unlike an approach that asks a second model to judge each call.

Key Result: 24 of 24 Would-Fail Runs Recovered#

The headline number is this repo’s own, produced by the fault-injection harness in faultinject.py and reproducible with uv run selfheal bench. The harness builds a seeded corpus that injects each of the four failure modes, then runs every scenario twice: once through a bare pipeline with no layers, once through the full stack. A scenario counts as recovered only when the bare run fails and the healed run succeeds. The denominator is the runs that would otherwise have failed, so the rate cannot be inflated by scenarios that never had a problem.

On that seeded corpus it recovered 24 of 24 runs, 100%, six per failure mode, with the seed fixed at 1234 so the result is reproducible.

Figure 2 - The actual benchmark report: The 100% hero, the four stat tiles, and the recovery-by-failure-mode table, rendered as a single self-contained HTML file with no external assets and both light and dark themes. Each of the four modes recovers all six of its seeded faults. This is the same report uv run selfheal report hands to a client, stamped with the date and an optional client name.

The per-scenario ledger is the honest backing for the headline. Every one of the 24 rows shows the bare run failing with a concrete error and the healed run recovering, so the number is not a summary statistic sitting on top of hidden aggregation. It is 24 individually verifiable before-and-after pairs.

Figure 4 - Every scenario, individually verifiable: The ledger behind the 100%. Each row names the injected fault, shows the bare pipeline failing with its real error, and shows the healed stack recovering. The hallucinated-name rows carry the exact UnknownToolError the model would have hit; the fuzzy match repaired each invented name back to a real tool before the call ever ran.

Delivery Forms: One Seam, Three Wirings#

The same stack interposes on the same seam no matter how an agent is built, so it ships in three delivery forms that all wrap the point where a tool name and arguments become a result.

Figure 5 - Three wirings, one stack: A custom loop wraps its tool caller directly, an MCP server adds the middleware to its chain, and a LangChain build wraps its tool list. All three route through the identical four-layer stack, so the recovery behavior is the same regardless of harness.

• Custom loop: build a ToolRegistry, stack default_layers, and call through the Pipeline. If you only have a bare (name, args) -> result function, wrap_tool_caller (and wrap_async_tool_caller for async callers such as an MCP client) wraps it directly with no registry.
• MCP: SelfHealingMiddleware plugs into a FastMCP server’s middleware chain and heals every tool call flowing through it. It is duck-typed against the FastMCP contract, so the mcp package is only needed to run the server, not to configure the middleware. This is the most portable form, because any MCP-speaking agent inherits the healing.
• LangChain: wrap_langchain_tools takes a list[BaseTool] and returns a healer you call wherever the agent dispatches tools.

Telemetry rolls up the same way across all three. collect reads the instances the pipeline already uses, so there is no extra instrumentation and no hot-path cost, and total_recoveries becomes a live gauge of every failure the stack absorbed that the model never had to see.

What It Does Not Do#

These caveats travel with every client-facing deliverable, because they are the difference between a defensible report and an overstated one.

• The 100% is on this seeded corpus, not a universal guarantee. It measures that each layer recovers the fault it targets across the corpus variants. Real workloads carry faults the harness does not model. The honest claim is “recovers the four modeled failure modes on the seeded benchmark,” never “makes any agent unbreakable.”
• The MCP form leaves result offload off by default. Swapping an oversized ToolResult for a summary means rewriting the result’s content blocks, so offload is opt-in there and enabled deliberately once the server carries the summary. Tool-name repair, loop breaking, and typed retry are on.
• Loop detection has false positives by nature. A legitimately retry-heavy task, such as polling a job until it is ready, looks like a stuck loop. That is exactly why the threshold is configurable and never hard-coded, and why a loop_exempt predicate can whitelist a known-polling tool outright.
• The 94% figure some talks cite is not this project’s number. It is monday.com’s unbenchmarked stage testimony from Interrupt 2026, and “variable advisory” is a speaker’s informal coinage, not a standard API name. This page leads with the repo’s own re-runnable 24/24 for exactly that reason.

Platform Portability#

The middleware is model-agnostic and harness-agnostic by design, because its signals are mechanical rather than tied to any one vendor. An invented tool name, an error class, a repetition count, and a payload size mean the same thing whether the agent runs on Claude, GPT, or a local model, and whether it dispatches tools through a custom loop, MCP, or LangChain. The MCP form is the most portable delivery vehicle, since any MCP-speaking agent, Codex, Copilot-via-MCP, OpenCode, or Claude Code, inherits the healing without a per-harness hook. We position it publicly as platform-agnostic agent resilience middleware.

As a Consulting Engagement#

The middleware is the instrument behind our agent-hardening work. A typical engagement installs selfheal into the client’s agent and wraps their tool seam with the form that fits their harness, with no model changes because the recovery is transparent. We then tune the thresholds to their workload, exempt their polling tools, set retry attempts to their SLA, and point result offload at a FilesystemStore so big payloads leave the context window and stay out of the re-fed history on later turns. We ship collect(...).as_dict() into their logging or tracing stack so the recovery rate becomes a live gauge instead of a hope, and we prove the before-and-after with selfheal report on the client’s own fault corpus rather than a borrowed number. The consulting frame is direct: we turn a demo-grade agent into a production-grade one by engineering the resilience the march-of-nines math demands.

It composes cleanly with its sibling projects across one reliability suite: TraceKit measures the token bill and grades the execution trajectory, ContextLedger cuts the bill, RubricGate grades the final artifact against a rubric, and the self-healing middleware keeps the run from dying at the tool seam in the first place. TraceKit tells you the run looped or blew its context; this layer is what stops the loop and offloads the payload before it does.

Technologies#

The throughline: an agent dies on a single bad tool call, and that call is never the model’s fault. This middleware sits on the seam between the agent and its tools, heals the four mechanical failure modes before the model ever sees them, and publishes its own re-runnable recovery number instead of borrowing anyone else’s.