An agent left running for six hours does one of three things. It drifts off the task. It takes a shortcut that looks like progress. Or it stops early, decides a half-finished answer is good enough, and reports success. None of those failures is fixed by a better prompt, and none of them is fixed by waiting for the next model. They are fixed by the system you build around the executor.
That is the whole argument of this article, and it is the argument the last eighteen months of our harness writing has been building toward. The agent is the engine. Reliability lives in the controls wrapped around it. There are seven of them, and this article walks all seven: Goal-as-contract, a split doer and judge Evaluator, two-layer Verifiers, an outer Loop, role-based Orchestration, Observability as a control surface, and session-mining Memory.
Figure 1 - The Seven Controls Around the Engine: The executor sits at the center and does the work. Everything that makes the work trustworthy sits in the ring: a contract-shaped goal, a judge that never sees the doer’s reasoning, verifiers staged cheapest-first, an outer loop that supervises, one model per role, observability you can act on mid-run, and memory that turns past failures into rules. Remove the ring and you have a demo.
One thing to say plainly before we go further, because this article’s own thesis is about not trusting unverified inputs. The seven-component decomposition we are using as the spine originates in a Latitude-sponsored explainer on the Prompt Engineering YouTube channel [1]. That is a practitioner teaching model, not vendor-neutral research and not a peer-reviewed framework, and we treat it as such. The durable authority sits elsewhere: two production systems shipped in 2026 converged on almost exactly the same anatomy without coordinating. LangChain shipped LangSmith Engine and published its architecture in an engineering post [2], and Anthropic’s Claude Science is in beta [3], launched 2026-06-30. When a sponsored explainer and two independently-built shipping products describe the same shape, the shape is worth taking seriously. The explainer alone would not be.
Why this is the capstone
Every article in this arc taught one piece of the ring. The Pi Comparison established that a coding agent’s behavior reflects its harness at least as much as its model. Delete the Bash Tool made the security case for constraining what the executor can reach. Your Agent Harness Belongs in Git argued that harness state has to be declarative and versioned. Managed Agents and Dynamic Workflows covered the runtime and the loop. The Orchestration Tax found the human ceiling on parallel supervision. Evals in Practice and The Verification Layer for Knowledge Agents built the judging half. The Context Engineering Stack and Two Hierarchies built the memory half. The Log Is the Agent made the case that the transcript is the asset.
This article is the assembly. Not a summary of those pieces, but the operating model they add up to when you run them together against one agent that has to survive unattended for hours.
There is a reason the assembly matters more now than it did a year ago. A Google Cloud Tech discussion put the shift in plain terms: when you hand-write code, most of the review happens as you write it, and with an agent that ratio inverts, so most of the review lands after the fact [4]. Those proportions are the speaker’s own conversational framing rather than measured data, and we treat them that way. The structural claim underneath is what holds up. The review burden did not disappear when agents started writing the code. It moved to the far end of the process, where nobody had built anything to catch it. The seven controls are what you build at that end.
It is also worth knowing that the engineering is worth something measurable. NVIDIA published a harness comparison holding the model fixed and varying only the harness design, and reported both higher accuracy and materially lower token cost from the better-engineered harness, with the framework and evaluation code released publicly [5]. Those are NVIDIA’s own self-reported results on their own harness, so treat them as a vendor claim that happens to be reproducible in principle rather than as an independent audit. The direction is the useful part: harness design is a performance variable, not overhead you pay for safety.
Control 1: The goal is a contract, not a prompt
A prompt tells an agent what to do. A contract tells it what “done” means and what it may not do to get there. The framework’s own framing is that the goal “is more than a prompt. It’s really a contract between you and the agent” [1], and a contract has four parts: the end state, measurable success criteria, the constraints the agent cannot break, and a spend budget.
The test is brutally simple. “Add a settings page” is a prompt. It licenses the agent to decide what a settings page is, which settings belong on it, and whether a page that renders but saves nothing counts as finished. “Match this design, persist every setting, and pass this test suite” is a contract. The second one can be evaluated. The first one cannot be evaluated at all, which means every downstream control in this article is inert until the goal is written properly.
Figure 2 - Weak Goal Versus Contract: The left column is a prompt an agent can satisfy in a dozen incompatible ways. The right column is a contract with four required parts. If you cannot fill in all four, the honest move is to stop and specify rather than to start the run and hope.
Two production observations sharpen this. Ankit Jain, speaking about why spec-driven development fails the same way waterfall did, made the diagnosis that intent lives in the back-and-forth of the agent session rather than in the spec, and gets discarded the moment the pull request opens [6]. His summary line is the one worth keeping: “Reviewers are reviewing the intent, not the diff” [6]. That is the goal-as-contract argument arrived at from the review side rather than the authoring side. If the contract was never written down, the reviewer has to reconstruct it, and reconstruction at review time is exactly where the process breaks.
Factory’s approach is the same idea given a shape you can build. Their missions pattern makes every outcome a structured hypothesis object carrying its own validation criterion, judged either deterministically or by an LLM [7]. The mission is the contract as a data structure rather than a paragraph, which means the loop and the verifiers can both read it without a human interpreting anything.
Control 2: Split the doer from the judge
The agent that did the work is the worst possible candidate to grade it. It has a full memory of why every decision seemed reasonable, and it will rationalize the path it already took. The rule from the framework is explicit: the evaluator “should be just looking at the initial specs and the final output and provides a verdict. It shouldn’t be sharing the same context with the agent that is implementing the work” [1].
Figure 3 - Execution and Judgment Are Separate Lanes: The judgment lane receives the contract and the artifact. It does not receive the executor’s reasoning trail, its intermediate attempts, or its self-assessment. That restriction is the mechanism, not a detail of it.
The restriction is the part teams skip. A verifier that reads the builder’s reasoning inherits the builder’s framing of what “done” meant, and confirms whatever the builder already believed. A hands-on build walkthrough on freeCodeCamp demonstrates the discipline in a running harness: at each milestone, a fresh verification agent is spawned that receives only the goal, the success criteria, and the declared invariants. The speaker’s rationale, from the walkthrough (captions lightly cleaned): “Spawning an independent L4 verify agent with fresh context. It only gets the goal, success criteria, and invariants, not our builder’s trail. Why? Because we need a separate checker to test” [8]. The verifier is further instructed not to assume the builder’s claims are true, and when its assessment disagrees with the builder’s self-report, a debug loop fires and a regression test gets added.
Both convergent production systems implement this. LangSmith Engine splits screening from investigation: a narrow Haiku-based screener subagent, dispatched across groups of roughly 20 traces, decides only whether a trace warrants a closer look and is explicitly barred from creating issues, while separate investigator subagents pull the full trace and do the deeper analysis [2]. In a LangChain-produced interview, Engine’s product manager described a further verifier sub-agent running a final light check to confirm an issue is real before it reaches a human [9]. Claude Science ships a dedicated reviewer agent whose only job is checking the other agents’ work, flagging incorrect citations, catching numeric claims that cannot be traced to a source, and spotting mismatches between code and the figure that code supposedly produced [3]. Two vendors, two products, same architectural move.
The same logic condemns a common shortcut. When the agent that wrote the feature also writes the tests meant to catch its mistakes, the test suite inherits the feature’s blind spots. Green checkmarks from a self-authored suite are the doer grading its own homework with a rubric it wrote.
KEY INSIGHT: Isolation is the mechanism, not the second agent. A judge that can see the doer’s reasoning is not a judge, it is a second opinion from the same mind.
Control 3: Verifiers in two layers, cheapest first
An agent’s “I’m done” is a claim. Verifiers are what turn a claim into evidence. The framework’s design rule is a two-layer staging: cheap deterministic checks run first and always, and expensive external checks run only on work that has already cleared the cheap layer [1]. The framing that makes it stick is a climbing anchor. The agent can feel as sure as it likes, and the anchor either holds or it does not.
| Layer | Cost | What runs there | Rule |
|---|---|---|---|
| Layer 1: deterministic | Cheap | Compile, type check, lint, unit tests, format and length checks | Runs first, always, on everything |
| Layer 2: external | Expensive | Benchmark against baseline, screenshot diff, held-out eval, LLM judge | Runs only on work that cleared Layer 1 |
Figure 4 - The Two-Layer Verifier Gate: Cheap checks reject most bad work at near-zero cost, so the expensive layer only ever runs on candidates worth the spend. Only what clears both layers counts as done. The framework’s own guidance is that good setups use deterministic checks as the baseline with agent review layered on top, never one instead of the other.
The strongest argument for layering is what happens when an agent meets a weak verifier. The SWE-Marathon benchmark authors disclosed a case where the task was to build a C compiler in Rust from scratch. One rollout shelled out to the system’s installed GCC binary and had the Rust program call it, so the output matched the reference behavior perfectly under any verifier that only checked output [10]. The catch was a layer most teams never build: an strace-based check watching what the agent’s process tree actually did, which spotted the forbidden subprocess call and zeroed the reward even though the partial test scores looked strong.
That is what a boundary of trust means in practice. It is not “run more tests.” It is having at least one channel that fails differently from the others.
We built our own version of this argument into a shipped tool. TraceKit grades an agent’s trajectory rather than only its output, and on a labeled fault set it caught 17 of 17 injected process faults that an output-only check could not see. Same principle: the artifact can be right while the path to it was wrong, and only a channel that watches the path will notice.
Control 4: The outer loop keeps the work alive
The loop that matters is not the tool-calling loop inside the agent. It is an outer loop that wakes the agent, compares its current state against the goal contract, and takes one of three actions: mark it complete, send it back with the specific failures named, or re-plan and escalate to a human. The framework’s phrasing is the useful one: the target is not “a long-running agent that is simply thinking for us,” it is “one that is taking short attempts supervised by this loop” that refines its approach when needed [1].
Figure 5 - Repeated Supervised Attempts, Not One Long Thought: The outer loop is the only control that catches early stopping, because early stopping looks like success from inside the agent. The loop is what asks “against the contract, is this actually done?” at an interval the agent does not control.
Gaurav Mishra of Amazon AGI Lab described the runtime-guardrail version of this for agents that drive a real browser rather than a repository, and his list is the most concrete inventory of loop-adjacent controls we have seen from a named lab: checkpointing and rollback, an action risk classifier, credential guardrails, an execution monitor, audit logs, and human handoff [11]. Two of those are pure outer-loop machinery. The execution monitor watches for loops, repeated clicks, and unproductive behavior during the run and intervenes. Human handoff overrides the model wherever its own confidence calibration is wrong, rather than letting it self-assess whether to continue.
Mishra’s framing of the domain shift is worth carrying: “A big realization has been that RL worked when the world was a game and IRL starts when the game fights back” [11]. His list of the six ways the real world fights back reads like a specification for the loop: partial observability, irreversibility, non-determinism, ephemeral authority, ambiguous success, and adversarial content. “Done often doesn’t mean successful” [11], and the loop is the thing positioned to tell the difference.
Control 5: Orchestration means roles, not a model
The single most expensive mistake in agent architecture is running one model for every job. The framework’s rule is to “stop thinking about the model, but rather think about the roles,” at which point “model choice becomes an architecture decision” [1].
| Role | Best-fit model | Note |
|---|---|---|
| Planner | Strongest reasoner available | A human reviews and sharpens the plan before it enters the loop |
| Executor | Fast, cheap coder | Highest call volume, lowest per-call cost |
| Evaluator | Capable but cheap judge | Runs in a context isolated from the executor |
| Vision review | Multimodal | Screenshot, layout, and rendered-output checks |
Figure 6 - One Model Per Role, One Human On the Plan: Planning is where human expertise pays the most, because an error in the plan propagates through every execution step downstream. The human checkpoint sits on the plan, not on every diff.
LangSmith Engine is the production instance. Its engineering post names the cheap end of the mix directly, a Haiku-based screener dispatched in bulk beneath a more capable main agent [2]. In the LangChain-produced interview, the product manager filled in the rest: Opus as the main agent, Haiku for the screener and verifier sub-agents, other vendors’ models swapped in and evaluated as a cost exercise, with the team profiling which pipeline segment drives cost and then hill-climbing against their eval suite when substituting a cheaper model for that segment [9]. That second half is a vendor-produced interview rather than a vendor-direct publication, and we mark it as such. The principle survives either way: model selection is an engineering discipline rather than a preference.
The ceiling on this control is human, not technical, and we measured it in The Orchestration Tax. Adding parallel agents adds supervision load faster than it adds throughput, and the binding constraint arrives well before the token budget does.
Control 6: Observability is a control surface, not a report
Nobody reads a six-hour raw transcript. The framework’s implementation detail is the part that makes observability usable: separate storage from presentation [1]. Raw logs and traces live in a searchable store the agent and its evaluators can query. A purpose-built dashboard renders tasks, costs, errors, screenshots, and key decisions for the human. The line to remember is that “observability is your control surface, not a report you read after the fact” [1].
Figure 7 - Storage and Presentation Are Different Products: The store is optimized for agents and evaluators to query. The dashboard is optimized for a human to notice one thing and step in. Conflating them produces a system where the human is expected to read machine output at machine volume, which is the failure this control exists to prevent.
Dan Farrelly’s layering argument explains why this control is worth building properly rather than bolting on. In his framing, prompts last weeks, models last months, “but execution can last years, if you do it right” [12], and the execution layer is “the system responsible for running your code reliably, managing how, when, or whether each piece of work completes” [12]. One of his three requirements for that layer is full-session observability covering tool calls, database errors, and permission failures, not only LLM calls. Observability built into the fast-decaying layers gets thrown away with them. Observability built into the execution layer survives the model swap.
There is a harder version of this argument, and it belongs here rather than in a safety section. Nathan Lambert, writing about a series of model misbehavior incidents, observed that “Our AI systems have scaled well beyond human oversight… state-of-the-art evals and monitoring are at a scale where only agents can monitor them” [13]. He also argued the lag is structural rather than a one-off operational failure at one company, since frontier labs “do not seem like they’re watching the models closely enough, due to a general frenetic competitive environment” [13]. If the organizations with the most resources and the strongest incentives cannot sustain manual oversight at their scale, a reader running agents in production should not plan to sustain it at theirs. Observability has to be engineered to be actionable, because it will not be read exhaustively.
KEY INSIGHT: If your observability plan requires a human to read everything, you do not have an observability plan. You have a backlog.
Control 7: Memory means mining sessions into rules
Past agent runs are training data most teams throw away. The seventh control closes that gap with session mining: scan recent runs for repeating patterns, the same mistakes, the same failed checks, the same wrong paths, and promote each pattern into an explicit rule in the agent’s configuration file [1]. The framework calls this “a naive version of recursive self-improvement” [1], and the naive part is a feature. It is a human reading failures and writing rules, which is auditable in a way an automated rewrite is not.
Figure 8 - Failures Become Rules, Not Folklore: The loop is deliberately slow and human-gated. A pattern has to recur before it earns a rule, and a human writes the rule. That is what keeps this control from becoming the ungated self-evolution the evidence section is about to warn against.
Jain’s version of this has a name we like: the AI slop registry, built by mining the last 1,000 review comments so that recurring feedback becomes a codified guardrail rather than something three senior reviewers happen to remember [6]. Recurring review feedback is a failure pattern with a paper trail already attached.
LangSmith Engine runs the automated production form. Its agent overview document is an agent-configuration-style memory file the main agent reads and updates on every run, fed by a hybrid of in-line updates during a run and background updates that fold in human feedback left between runs [2]. LangChain’s own post puts the payoff modestly, that Engine “has already changed how we improve our own agents internally” [2]. The stronger claim, that running Engine against Engine’s own traces has become one of the primary ways the team finds improvements, comes from the same vendor-produced interview [9]. Session mining is the single-agent, human-paced version of exactly that.
Three 2026 results that line up on one axis
This is the part that separates an operating model from a repackaged framework. Three independent results published in 2026 point the same direction, and the honest sentence is that they line up on one axis rather than that any one of them resolves the others.
Figure 9 - Three Results, One Axis: A negative result on ungated harness self-evolution, a benchmark finding that the optimizer separates more than the harness it acts through, and a positive result from a self-evolution loop that gates every accepted edit on held-out performance. Different experiments, different teams, one shared variable.
The negative result. Researchers at the Allen Institute for AI and the University of Washington, with independent collaborators, evaluated automatic harness evolution and found it did not pay [14]. Their abstract states it directly: “automatic harness evolution does not consistently outperform simple test-time scaling methods and exhibits limited generalization” [14]. The table numbers support the careful version of that claim, and the careful version is the one to make. Averaged across the three models the paper tests (Claude Opus 4.6, GPT-5.4 and GPT-5.4 mini) on Terminal-Bench 2.1, simple parallel sampling scored 72.3, the harness-evolution arm scored 67.4, and the unmodified baseline harness scored 68.2 [14]. That arm landed below the baseline. A second harness-evolution variant the paper tests, harness scaling, did better at 71.8 and posted the best single Claude Opus 4.6 result in the table, yet still did not beat plain parallel sampling [14]. Letting the model rewrite its own harness did not reliably beat leaving the harness alone, and plain repeated sampling of the fixed model beat every harness-modification strategy on average. The mechanism is the interesting part: an evolving harness memorizes file paths and hardcodes benchmark-specific fixes, so it overfits the eval it is being scored against.
A note on precision, because this article argues for engineered verification and this is a case of it working. The paper’s held-out transfer result is often relayed as a 0.6-point gain. That 0.6 is a two-model average, and the spread underneath it matters: one model gained 1.2 points and the other gained exactly 0.0 [14]. The relay that carried those numbers to us stated the average as though it were a per-model result. We caught it by reading the tables, which is precisely the kind of check this whole article is arguing for. That is worth admitting in public rather than quietly fixing.
The benchmark result. HarnessOpt-Bench evaluated LLMs at the task of optimizing a harness, running ten core configurations (five optimizer models across two harnesses) over four tasks and 111 scored runs [15]. The headline finding is that the optimizer separates results more than the harness it acts through: changing the optimizer model moves gain by 0.142 on average, while holding task and model fixed and changing the harness moves it by 0.079, so the model contrast is about 1.8x larger [15]. One deflating detail belongs alongside that, since leaving it out would make the citation dishonest. The largest single gain, 0.49 on GAIA, started from a non-functional stub with a measured-zero baseline [15]. The paper’s own conclusion is modest, that optimizer models separate more than the coding harnesses they act through, that native harnesses are not consistently superior, and that gains vary substantially across tasks and seed regimes [15].
The positive result. The AutoDesign authors froze the LLM and built an external meta-optimizer that recursively rewrites the harness, restricted to five editable components: context and memory, tools and specs, execution runtime, orchestration, and evaluation and feedback [16]. No gradient training is involved. A coding agent acts as the optimizer and proposes a bounded update each iteration, so this is an inference-time loop rather than a training run. The part that matters is the acceptance rule. An edit is committed only if it improves the training set and does not regress a held-out development set the optimizer never sees when constructing its update proposal [16]. AutoDesign adopts that acceptance gate from earlier work rather than inventing it, which strengthens the point rather than weakening it: the gate is a reusable control, not one paper’s trick. Self-evolution with a generalization gate on every accepted edit worked in AutoDesign’s own domain. That is a different task and a different benchmark from the AI2/UW experiment, which is exactly why we read the pair as an aligned axis rather than as a controlled comparison.
Those five editable components are a near-exact structural echo of the seven-control anatomy, reached independently by a research group solving a design-generation problem rather than a coding-agent problem. We read that as convergence, not as proof.
Two caveats travel with this section and neither is optional. All three of these results are v1 arXiv preprints, unrefereed at the time of writing. AutoDesign carries seven author institutions, so we attribute to “the AutoDesign authors” and name none of them. If their 64.0% human-preference figure comes up elsewhere, it is a Bradley-Terry point estimate with a 95% interval of 55.2% to 77.8%, not a raw win rate.
There is also a reason from the training side that the negative result had to come out the way it did. Dex Horthy’s argument is that no amount of harness engineering solves a model-training issue, since SWE-bench-style reinforcement learning rewards test-pass rather than maintainability. His line is the compact version: “if the model knew what good code looks like, it would probably write it in the first place” [17]. A model optimized against a test-pass signal will optimize a harness against that same signal when you hand it the harness.
Figure 10 - The Difference Is One Gate: The left flow commits any edit that helps the score it is being measured on, which is how a harness memorizes its own benchmark. The right flow requires every accepted edit to also hold on data the optimizer never saw. That single AND is the engineered control, and it is the one a reader can actually build.
One sourcing note that is on-thesis rather than housekeeping. A single 2026 industry report from Faros AI reached our research through three separate speakers, each presenting its figures as their own supporting evidence, one of whom this article cites elsewhere [6]. That is one report relayed three times, not three independent confirmations, and it is the dominant sourcing failure in this domain. Rather than cite a figure through a relay, we use none of that report’s numbers here. Relay collision is the human-scale version of the memorization problem the AI2/UW paper found in machines: a number that circulates enough starts to feel independently confirmed.
KEY INSIGHT: Ungated self-evolution is the failure mode, not self-evolution. The fix is not distrust, it is a held-out generalization gate on every accepted edit.
What still breaks, and what catches it
None of the seven controls makes the hard problems disappear. What they do is give every failure mode a named owner.
| Still breaks | Caught by |
|---|---|
| Shortcutting to a passing result | Verifiers, especially a channel that fails differently from the others |
| Stopping early and reporting success | The outer loop, comparing state against the contract |
| A weak plan that propagates downstream | Human review of the plan, inside orchestration |
| Overfitting the thing being measured | Held-out evaluation in verifier Layer 2 |
| Stale or missing context | Memory, via session mining |
| Ambiguous success on irreversible actions | Human handoff, plus checkpointing before risky state |
Figure 11 - Every Failure Has an Owner: The value of the framework is not that the failures stop. It is that when one happens, you know which control was supposed to catch it and can go strengthen that one instead of rewriting the prompt and hoping.
The honest limit sits underneath the whole table, and Factory’s Eno Reyes stated it better than we can: “you need to validate the validators… it’s kind of turtles all the way down” [7]. Verification is not solved. It is a regress, and engineering it means choosing where to terminate the regress deliberately, at a level you can defend to whoever asks. Factory’s own answer is a governing principle worth adopting, that “agents succeed when they have non-human deterministic feedback” [7], and they gate work behind an agent-readiness check covering unit, end-to-end, integration, and fuzzy tests, test coverage, development environment reproducibility, flakiness detection, standardized metrics and logging, AST parsers, and code owners [7]. That is a deliberate termination point. It is not the bottom turtle, and it does not claim to be.
One more piece of honesty, and it cuts against the pessimistic reading of this whole article. Mishra described the harness relationship as temporary rather than permanent: “Early on, our harness is really strong… and over time, the model becomes better and better, and the harness becomes thinner and thinner” [11]. The controls are scaffolding sized to the current gap between what the model does reliably and what the job requires. They are not a permanent verdict on the model. They are also not optional while the gap exists, and his other line names exactly why: “The difference between a demo and a product is what happens after the first click, first failed click” [11].
The operating model: seven moves for Monday morning
Here is the build order, and the order matters more than any individual item.
- Start small and cheap. Prove the system on a task you can check in two minutes before you extend the time horizon to two hours.
- Write the goal as a contract. End state, measurable success criteria, hard constraints, spend budget. If you cannot write all four, stop here rather than starting the run.
- Separate the executor from the evaluator. Isolated contexts, and the evaluator sees the contract and the artifact only.
- Define the verifiers before you start the loop. Verifiers written after a loop is already running get written to match whatever the agent is already producing.
- Run deterministic checks first, agent review on top. Never the judge instead of the tests, and never the expensive layer before the cheap one.
- Require proof artifacts. Logs, screenshots, actual diffs. A claim of done with no evidence attached is not a claim, it is a mood.
- Mine sessions into rules. Recurring failure patterns get promoted to explicit configuration, and a human writes the rule.
Figure 12 - The Build Order Is the Advice: Steps 2 through 5 are ordered by dependency, not preference. A loop without verifiers runs forever, verifiers without a contract have nothing to check against, and an evaluator sharing context with the executor is a formality. Do them out of order and you get the appearance of the framework without its effect.
The one that teams skip most often is step 4. Verifiers feel like the last thing you build, since they are the last thing that runs. They are the first thing you should design, since they are the only artifact that defines what the loop is looping toward.
Conclusion
The sharpest thing to say about long-running agents in 2026 is not that self-improvement failed. It is that ungated self-improvement failed, and the fix has a name and a shape. Hand a model its own harness with only its benchmark score as feedback and it memorizes the benchmark [14]. Gate every accepted edit on a held-out set the optimizer never sees and the same class of loop works [16]. The variable is the gate. That is an engineered control, and a reader can build one this week. “Do not trust the agent” is not a control, it is a mood, and it does not survive contact with a deadline.
Everything in this article follows from that distinction. A contract-shaped goal is a gate on what “done” is allowed to mean. A separate-context evaluator is a gate on who gets to declare it. Two-layer verifiers are a gate on evidence. The outer loop is a gate on time. Role-based orchestration is a gate on which judgment call runs on which model, with a human on the plan. Observability is the gate you can see through while the run is still going. Session mining is the gate that gets stricter every week, since a failure that recurs becomes a rule. Seven gates around one engine, and the engine keeps getting better while the gates keep the gap survivable.
We build these for a living, and each of the seven controls maps to something we have shipped and can show you. TraceKit instruments controls 2, 3, and 6, reading the transcripts a client’s agents already write to disk and turning them into a cost-attribution curve and a reliability scorecard, with the agent-trace-grader inside it scoring trajectories rather than only outputs. RubricGate is control 2 as a portable tool: a fresh-context grader that sees only the rubric and the artifact, runs a deterministic tier before the judge tier, and loops the producing agent on the gaps until the deliverable clears. Both are delivered through our agent reliability audit, a read-and-grade pass over agents you are already running rather than a rebuild.
If your agents run unattended and you cannot currently answer what they cost or whether they behaved, those two questions are already answerable from data sitting on your disk. That is the audit, and it is the honest offer: we measure what your long-running agents actually do, map each failure we find to the control that should have caught it, and hand you the list. No rebuild required to find out where you stand.
References
[1] “Stop Building AI Agents the Old Way,” Prompt Engineering, YouTube (Latitude-sponsored), 2026. https://www.youtube.com/watch?v=ju7R6jer6_M
[2] LangChain, “How We Built LangSmith Engine, Our Agent for Improving Agents,” LangChain Blog, 2026. https://www.langchain.com/blog/how-we-built-langsmith-engine-our-agent-for-improving-agents
[3] Anthropic, “Claude Science: An AI Workbench for Scientists,” Anthropic News, Jun 2026. https://www.anthropic.com/news/claude-science-ai-workbench
[4] “Reviewing Code from AI Coding Agents,” Google Cloud Tech, YouTube, 2026. https://www.youtube.com/watch?v=ABC7ifp8-Uo
[5] R. Silveira Cabral and P. Furgale, “Six Agent Harness Capabilities for Higher Model Performance,” NVIDIA Technical Blog, Jul 2026. https://developer.nvidia.com/blog/six-agent-harness-capabilities-for-higher-model-performance/
[6] A. Jain, “How to Kill the Code Review,” Aviator, AI Engineer, YouTube, 2026. https://www.youtube.com/watch?v=YgEv7IQzGdM
[7] E. Reyes (Factory), interviewed in “The best AI agents cost less than you think,” LangChain, YouTube, 2026. https://www.youtube.com/watch?v=HbUznYhKFOc
[8] A. Singh, “System Design for AI Agents: Building a Multi-Agent PR Reviewer,” freeCodeCamp.org, YouTube, Aug 2026. https://www.youtube.com/watch?v=iqRcGCah0Kw
[9] B. Tanneyhill (LangChain), interviewed in “The best AI agents need less code than you think,” LangChain, YouTube (vendor-produced interview), Jul 2026. https://www.youtube.com/watch?v=YqjR4vQwbTc
[10] R. Desai, “SWE-Marathon: Evaluating Coding Agents at Billion-Token Scale,” Abundant AI, AI Engineer, YouTube, 2026. https://www.youtube.com/watch?v=Rx8f05JI_WA
[11] G. Mishra, “From RL to IRL,” Amazon AGI Lab, AI Engineer, YouTube, Aug 2026. https://www.youtube.com/watch?v=Cc0_nyxROBA
[12] D. Farrelly, “Your Agent Architecture Has a Half-Life of 6 Months,” Inngest, AI Engineer, YouTube, 2026. https://www.youtube.com/watch?v=X1kp-ABIIxQ
[13] N. Lambert, “Lessons from the Hacks,” Interconnects, Aug 2026. https://www.interconnects.ai/p/lessons-from-the-hacks
[14] Y. Wang et al., “Rethinking the Evaluation of Harness Evolution for Agents,” Allen Institute for AI, University of Washington, and independent collaborators, arXiv:2607.12227 (v1 preprint, unrefereed), Jul 2026. https://arxiv.org/abs/2607.12227
[15] V. Ursekar et al., “HarnessOpt-Bench: Evaluating LLMs at Harness Optimization,” arXiv:2608.06301 (v1 preprint, unrefereed), Aug 2026. https://arxiv.org/abs/2608.06301
[16] Y. Luo et al., “AutoDesign: Meta-Harness Optimization for Long-Horizon Agentic Design,” arXiv:2608.13560 (v1 preprint, unrefereed), Aug 2026. https://arxiv.org/abs/2608.13560
[17] D. Horthy, “Harness Engineering Is Not Enough: Why Software Factories Fail,” HumanLayer, AI Engineer, YouTube, 2026. https://www.youtube.com/watch?v=Ib5GBkD555M
Building production AI, or modernizing a legacy system?
That is the kind of work we do at Dotzlaw Consulting. Book a free 20-minute intro call and tell us what you are trying to build, or what is slowing you down.