The Failure Was Not the Fix. It Was the Absence of a Check#

That 8-day life is the whole argument of this article. The fix was correct, measured the day it shipped, and verified by running the loader rather than by reading a file. What it never got was a number for how long it would keep working, and no commit anywhere says “expand the injected context.” The harness crossed back over the line invisibly, the way it crossed it the first time, then stayed there for 38 days because nothing re-measured it and nothing raised an alarm. We have since closed both of those gaps with machinery rather than intention, which is what the closing section of this article describes.

Resist reading this as a cost story. The token saving from the original fix was about 1,218 tokens, roughly 16%, which is noise. The cap had already bounded the spend. The win was correctness: for an unknown number of months, the agent believed it had seen something it had not seen, and then it believed the same thing again for 38 more days. Silent truncation is worse than an error, because an error at least announces itself.

KEY INSIGHT: A harness cleanup is a correctness exercise wearing a cost exercise’s clothes. Justify it on tokens and you will stop as soon as the tokens look fine, while the lie in the comment stays exactly where it is.

The method we ran is Nate B Jones’s, published after he audited his own setup: 66 reusable skills and 172 instruction-related files, one content route weighing 18,000 words, the same authorship rule living in 15 different top-level skills, and only 6 of those 66 root skills carrying any detected local eval [1]. His line for the phenomenon is the best one available: “Harnesses grow barnacles like ships” [1]. The object he means is everything wrapped around the model: instructions, project files, saved prompts, memory, skills, tools, permissions, and any checks that run [1].

Every number there is one practitioner’s self-report about one harness. What follows is ours, measured on our own repository, which is the only reason it is worth your time. We sell harness work. Running the instrument on ourselves first, and publishing what it got wrong, is the price of that.

Figure 2 - 8 good days, 38 silent ones: The remediation bought about 4,855 characters of headroom under a 30,000-character cap, and ordinary growth consumed all of it by 2026-07-24. The re-measurement 46 days on did not catch a fresh regression. It caught one that had already been running for 38 days.

The 46-day window is also long enough to watch the rest of the harness move. The pool of skill, agent, and command descriptions that loads every session, so the model knows what it can invoke, grew from 26,582 characters to 28,207 over the same interval. Roughly 6% on the largest always-loaded item in the harness, with nobody deciding to grow it. Both of the two biggest always-loaded line items drifted upward, and one crossed a hard limit in the first 8 days.

Now the counterweight, which an honest article cannot leave out. Control counts barely moved. We had 21 project skills on audit day, 20 after the retirement that pass recommended, and 21 again today. Commands were 26 then and 26 now, and the user layer was 14 skills, 11 commands, and 12 agents then and now. What changed was the content inside those controls and the payload the hooks inject.

Figure 3 - The count is the wrong number: Skill and command counts were identical across 46 days, while the description pool those controls feed grew about 6%. An audit watching the inventory count sees a harness that did not change.

One convergence is worth reporting, carefully. Jones measured 27,000 description characters on his harness [1], against our 26,582 in July and 28,207 in August. Two unrelated practitioners, two unrelated harnesses, three measurements, all within about 6% of each other. That is the closest thing to external corroboration this method has, and it corroborates the measurements only. He compares his figure against a discovery budget he attributes to another vendor’s product, and that budget has no source page behind it.

Principle 1: Map the Harness Before You Clean It#

Jones’s first principle is “You map the harness before you clean it” [1], and the mapping step is six questions asked of every single control: “Where does this control live? When does it load? What job does it do? Who owns this? Is there any evidence that it still helps? What problem can it create if it’s misused?” [1].

Most summaries drop the sixth question. Keep it. The first five produce an inventory, and the sixth turns that inventory into a risk assessment, which is what stops you deleting something that turns out to have teeth.

Figure 4 - One row, six columns: Every control gets the same six questions. The sixth, what breaks if this is misused, is the one paraphrases drop.

One column has no obvious way to fill it in, and it is the one that matters most: when does it load. Guessing there is how an audit produces confident false findings. Our own first pass read file sizes off disk and reported a cost roughly 10x the truth, because it never ran the loader that was capping the payload. Measure the loader, not the file.

Our regression is that same lesson aimed at content instead of cost, and it explains the truncation better than “the knowledge base grew” does. When we finally opened the map rather than measuring it, 134 of its headings turned out to be dated ingest-batch log lines reading like “2026-08-27 36-file research triage (23 YouTube + 13 blogs)”. Those 134 headings accounted for 47,958 of the map’s 68,961 characters, 70% of a payload whose entire job is navigation. The map was carrying an ingest log, and it had been carrying one in July too, when we measured the payload and never read it. Map before you clean is not a metaphor. We had not looked inside our own map.

For the vocabulary of what loads when, the strongest source is not a practitioner. Anthropic documents Claude Code’s own prompt as three layers, ordered by how often each changes [2]:

Two more items sit outside the prompt text but inside the cache key: the model, and the effort level [2]. What that ordering costs you is stated on the same page: “A change to the conversation layer leaves the system prompt and project context cached. A change to the system prompt invalidates everything, because all later content now sits behind a different prefix” [2].

Read as an audit instruction, that adds something the six questions only imply. A control is not expensive because it is long. It is expensive because of where it sits in the prefix and how often the things above it change, a cache-key question rather than a word count.

Figure 5 - Ordered by what changing it costs: Anthropic’s own layer table sorts the prompt by change frequency. A conversation-layer change leaves everything below it cached. A system-prompt change invalidates all of it.

The clearest illustration is a change most people file under the wrong layer, since switching models feels like a model decision: “Switching with /model means the next request reads the entire conversation history with no cache hits, even though the content is identical” [2]. Jones’s own framing then tags each mapping row with one of four buckets people conflate: the model, the harness, the project context in files, and the conversation.

Principle 2: Blame the Layer, Not the Model#

Principle 2 is “Blame the right layer,” and the diagnostic question is “Did the model fail or did the surrounding setup fail?” [1]. Jones ran the same job against the same model under two harnesses, one compact and one thick. The thick setup produced richer analysis and “also failed actual delivery requirements twice. One result broke the JSON, another broke the word limit,” while “The compact setup finished correctly three times out of three” [1]. Treat that as an anecdote, because that is what it is: one practitioner, one job, three runs per arm.

The durable part is the loop underneath. Blame the model for a harness-caused failure, add an instruction to compensate, and the harness is thicker because of a problem it caused itself, which is how every bloated setup got built.

Figure 6 - Richer output, broken delivery: One practitioner’s single comparison, not a benchmark. The thick harness wrote better analysis, then broke the required JSON once and the word limit once.

Our own version of principle 2 is less flattering and more useful. Across three passes, our audit made the same class of error six times, reading an artifact and reporting an effect. It inferred a token cost from a file size. It counted grep matches and called them rule copies. It read hook scripts and called them hard locks. It ran a hook standalone, watched it print a denial, and called the layer fixed. It wrote 18 unit tests, watched them pass, and called the hook repaired. It credited a measured token drop to the wrong cause. One error was of a different class: the first pass never inventoried the user layer at all, so a third of the harness was missing from a report that presented itself as complete.

KEY INSIGHT: An artifact is evidence of intent, never of effect. A script printing deny is an opinion. A settings file naming that script is a claim. A green unit test is a claim about the test harness. Only the system refusing to run the thing is enforcement.

Principle 3: One Rule, One Home, One Owner#

Principle 3 is “One rule, one home, one owner” [1]. Jones found “versions of the same authorship and source rule in 15 different top-level skills” [1]. 15 copies of an important rule do not buy 15 times the enforcement. They buy 15 places where it can drift out of sync with itself, so one copy gets corrected after a failure and 14 do not.

Our audit went looking for the same shape and found our no-em-dash writing rule mentioned across 21 files, 60 occurrences in total, which the first count reported as roughly 24 homes for it. Reading all 60 dropped the real number to somewhere between 10 and 12 actual restatements, and a wholesale strip would have deleted working enforcement. The occurrences that were not restatements had to stay: verification steps stricter than the generic hook, and rules governing surfaces no checker can reach, meaning text rendered into an image by an image model, and a reviewer’s own report about the rule.

Then the harder finding. Our top-level CLAUDE.md did not state the rule at all, so there were 10 to 12 restatements of a rule with no owner and nowhere to consolidate to. Principle 3 assumes an owner exists. Often the owner has to be created first, and creating it is the real work.

Figure 7 - Count rules, not matches: 60 occurrences across 21 files, of which 10 to 12 were true restatements. The rest were verification steps and uncheckable surfaces, and every one had to survive. The owning file did not state the rule at all, so the consolidation target had to be written first.

Principle 4: Load Specialist Knowledge When the Work Needs It#

Principle 4 is “Load specialist knowledge when the work actually needs it” [1]. Jones found “six different editorial guides that were loading whenever one writing skill ran” [1], each relevant to exactly one sub-task, all six arriving for every task. The library was not the problem. The load timing was.

Anthropic’s documentation confirms what a lazy-load buys: “Skills and commands inject their instructions as user messages at the point of invocation. Nothing earlier in the conversation changes” [2]. Skill bodies genuinely are on-demand. Our audit’s first pass wrote a version of that conclusion and it was the single most wrong thing in the report: our 21-skill count had never been the problem, it said, since skills load on demand. Half right, and the wrong half is the expensive one. The bodies load on demand. The description of every skill, agent, and command is injected every session so the model can decide what to invoke, and on audit day that description pool was the largest always-loaded line item we had.

Figure 8 - What arrives before you type: 20,900 tokens of standing cost, measured by running each loader rather than reading each file. The largest line item is the description pool, which no earlier pass had counted at all.

Two omissions caused the undercount: the user layer, which the first pass never looked at, and the description pool, which no pass had counted. Together they made the report’s figure wrong by 3.3x. The honest close matters more than the number. 20,900 tokens of standing cost is not alarming for a modern context window, and no line item in that table is wasteful enough to justify surgery. The reason to know the figure is that we had previously believed it was 6,300. A harness that misreports its own cost by 3.3x cannot be reasoned about at all, and that is the defect, not the 20,900.

Control count, then, is not free after all, and a reader who takes away “add all the skills you like, they are lazy-loaded” has taken the wrong lesson. The same discipline applied to always-loaded rules is the argument in Context Engineering for Claude 5: The Six New Rules [3]. The retrieval-and-compression side sits in The Context Engineering Stack [4].

Principle 5: Hard Requirements Need Hard Checks#

Principle 5 is “If you have hard requirements, they need hard checks” [1]. A 50-word limit and a required JSON shape have binary answers a machine can test, and expressing them as prose reminders makes the harness heavier and the guarantee weaker at once.

We agreed with this so thoroughly that we had already written nine hook scripts. Then we checked whether any of them ran.

Six of the nine were referenced by no settings file at all, project, local, or user. Written in the initial scaffold months earlier and never installed, they had never executed once, and the one that mattered blocks destructive shell commands, force-pushes to main, and writes to a read-only backup. Our settings.local.json meanwhile carried a blanket Bash(*) entry that defeated the careful allowlist in settings.json. Nothing stood between an agent and a destructive command, and the repository looked well defended.

Wiring them did not fix them. In a live session we provoked three of the four controls that were supposed to bite: a Bash call containing a DROP TABLE statement ran normally, a Write to a .env file created the file, and a markdown file written with a real em dash produced silence. The scripts fired every time and their output was discarded, because a PreToolUse decision has to arrive nested under hookSpecificOutput and a bare top-level decision object is ignored. Every hook from the original scaffold carried the bug, while the later session-start injector used the correct shape and had always worked, so the bug belonged to the scaffold.

Two more faults hid behind the schema, and neither was findable by reading code. Every hook decoded its input with the machine’s locale encoding rather than the UTF-8 the runtime sends, so we had built an em-dash detector that could not physically receive an em dash. For the two permission guards that was a security fault, since a decode error crashes a PreToolUse hook and a crashed hook fails open. The lint hook had a third fault of its own: it called pnpm by bare name, which cannot resolve on Windows, and swallowed the error.

Stack those and one 80-line lint hook was a triple no-op: unwired, then wrong schema, then unable to find its own linter. Three independent faults in one file that had never once executed its purpose.

Figure 9 - Three faults, one file, zero output: The lint hook was never wired, then emitted a decision shape the runtime discards, then could not find its own linter. Each fault presented as no output, which reads exactly like clean.

All four are now repaired and proven by live provocation: a real destructive command refused, a real .env write blocked, a real em dash caught, a real lint failure reported.

Figure 10 - Only the last one counts: Three of these four are artifacts that represent behaviour. One is behaviour. An audit that credits any of the first three has manufactured a false finding.

The sharpest lesson came from the unit tests. All 18 passed over a hook still broken in production, because they forced a UTF-8 decode the real invocation never performs. A test that constructs a friendlier environment than production tests the test, not the thing. Encoding, PATH, and environment all differ between the two, and every one of those differences has hidden a real bug here, so provoke every control the way the system calls it. Whether your skills have any evals at all is a separate and equally uncomfortable question, and the subject of our companion piece Don’t Ship Skills Without Evals.

Two findings belong here as the honest counterweight. A control with teeth will bite its owner: the commit that landed these repaired hooks was blocked by one of them, because the message quoted the destructive pattern verbatim to describe what the guard now catches. A permission guard sees a string, and the string that must never run and the string naming it are the same string. That false positive is the price of the guarantee. Two of our nine scripts also remain deliberately unwired, one gating on an environment variable a subagent probe proved does not exist, since the appearance of enforcement is worse than a gap you can price.

Principle 6: Build for the Model and the Product Surface#

Principle 6 is “Build for the model and the product actually doing the work” [1]. A coding CLI, a chat product, and a raw API expose different tools, load mechanics, and checkable guarantees. What a task requires does not change across surfaces. What you can enforce does. A rule set validated in a chat UI cannot be assumed to transfer to a coding agent, and the mapping column that catches this asks which surface a control silently assumes.

Why Cleanup Has to Recur#

The 38 silent days at the top of this article are why the method cannot end at a cleanup. Two habits close that loop from outside, and a third, which we come to last, closes it from inside the harness.

The first is prevention. Garry Tan, speaking at the AI Engineer conference, gives it a test: once a task has been done well, capture it as a reusable skill file, because “if you have to ask for something twice, you failed” [5]. His stakes line is the better quote: “the organization that captures what it learns like this gets smarter every single day. The one that doesn’t wakes up every morning with amnesia, no matter how good the model is” [5]. He has restated the practice as a short loop on X: capture the task, add it to cron, then gate it with evals and integration tests [6].

Take the tension seriously, since it cuts against this article’s own thesis. “Skillify everything” is exactly how you end up with a 66-skill harness that needs an audit. Tan’s rule is not “capture every task.” It is never answer the same question twice, and the discipline lives entirely in the word twice. A task done once and not repeated should not become a skill file at all, and drawing that line separates a harness that compounds from one that merely accumulates.

The second habit is automation, and the strongest published account comes from Boris Cherny, who created Claude Code. In a post titled “A weird experiment I’ve been trying the last few weeks,” he reports running Claude on a daily schedule against Anthropic’s own applications, with a countable outcome: 388 pull requests opened across those repositories in the first few weeks, 180 of them merged, a merge rate of about 46% [7]. He names the routines individually, and two of them are easy to confuse. A dup unifier finds similar abstractions and opens unified pull requests, while an abstraction police routine fixes leaky ones. Two routines, two jobs. The whole thing runs through one dedicated Slack channel, and some routines needed a few days of tuning [7].

Hold onto that pairing, because the spoken version collapses it. On stage at Y Combinator, Cherny put the practice at 20 or 30 routines running daily and half-remembered the abstraction police name while doing it [8]. Treat the scale figures as his approximations and the routine names as what the written post says.

Three things transfer, and they are the whole graft:

• A routine is one sentence, not a framework. As published, each routine is a single stated job rather than a procedure, and none specifies a method. The dead-code remover deletes unreachable code, and goes as far as adding logging to code it suspects is dead so a later pass can confirm it [7].
• The output is a pull request, not a commit. Every routine ends at a human review gate [7], which keeps this from being reckless and runs into the same human ceiling we measured in The Orchestration Tax [9].
• The infrastructure is a timer. Cherny reports a daily schedule and one Slack channel [7], with no platform and no orchestration product in the loop, which is why a timer plus a one-line prompt is reproducible on a client codebase the same afternoon.

Figure 11 - The loop that keeps a margin from closing: Capture discipline stops accumulation, the audit finds what accumulated anyway, and a scheduled routine repeats the cutting on the chores worth repeating.

The reusable idea is not the routine list. It is the selection problem underneath it: which chores in your harness and your codebase are worth repeating forever. Copying somebody else’s routines is one more way of adding controls nobody chose, so the ordering runs audit first and automate second.

KEY INSIGHT: Automate the cleanup only after the audit has told you what to clean. A scheduled routine you did not choose is harness bloat on a timer.

The Fix That Holds Is an Alarm#

We repaired the hook on 2026-08-31, the same day the re-measurement caught it, and the shape of that second fix is the point the first one missed.

Three changes went in that day. The dated batch headings are no longer pasted into the map, which removed 70% of the payload at a stroke and cost nothing, since an ingest log was never navigational. The cap moved from 30,000 to 40,000 characters. Overflow now writes a warning to stderr and prepends a banner to the injected context, so a session that receives a cut map is told it received a cut map. The injection is 28,824 characters, all 26 sections arrive, and there are 11,176 characters of headroom.

Two more changes followed, and those are the two that answer why this will not happen again. The first moves the check upstream, from where the symptom appears to where the change happens. A PostToolUse hook now measures the session-start injection whenever a file that feeds it is written, and reports at 85% of the cap or on truncation, in the turn that caused the growth, silent otherwise. All three of its branches are tested: warn, truncated, and healthy-silent. Timing is the entire value here. The payload crosses the cap through content growth with no code change, so there is no commit and no diff for anyone to review, and a stderr warning speaks only to whoever starts the next session and reads stderr. This hook is the difference between loud eventually, to whoever looks, and loud at the moment of the change, to whoever made it.

The second puts the re-measurement on a schedule. A script now re-measures the mechanically checkable findings a previous audit closed, 8 of them, exiting 0 when all hold, 1 when one has regressed, and 2 when one is approaching a limit. It is wired to run daily in CI, and on any change to the hooks, the settings file, or the two files that feed the map, so the result is acted on by a scheduler instead of by a human remembering to look. All 8 hold today. They cover the session-context budget, whether every map section actually arrives, whether truncation is still loud in both channels, whether the ingest-log filter and the em-dash hook’s coverage of content and drafts survive, whether a blanket Bash permission has returned, whether the load-bearing hooks are still wired in settings.json, and the working-tree state.

Five changes, and they divide cleanly. The heading purge and the higher cap bought margin, the same kind of thing the July fix bought, and that margin will close again on a date nobody has computed. The loud truncation, the write-time hook, and the scheduled re-check do not expire, because they change when you find out rather than how much room you have. A silent-truncation defect is now a visible one, caught in the turn that causes it, and a visible defect gets fixed in the week it appears rather than 38 days later.

That is principle 5 applied to our own remediation. A hard requirement, the map arrives whole, had been expressed as a code comment asserting that it would. A comment is a prose reminder. The stderr line is a check, the write-time hook is that check moved to the moment of the change, and the scheduled re-check is the audit itself turned into a property.

KEY INSIGHT: The durable half of a context fix is never the bigger number. It is the alarm that fires when the bigger number runs out.

Conclusion#

The method holds up. Map before you clean, blame the layer before the model, give every rule one home and one owner, load specialists when the work needs them, convert hard requirements into hard checks, and design for the surface actually doing the work. Five of those six surfaced something real when we ran it on ourselves. The sixth we could only reason about, since we audited one surface rather than several, and it is the principle that marks where our findings stop transferring.

The practical version of this article is shorter than the article. Run the map, measure the loader rather than reading the file, and read what is in the payload rather than only how big it is. Count rules rather than grep matches. Provoke every hard lock the way the system calls it, and treat silence as failure until you have proven the check can speak. When you raise a limit, work out the date the new limit expires, then give the limit a voice so it tells you itself. Wire that voice where the change happens rather than where the symptom shows up, since the turn that grew the payload is the only one with somebody there to act on it. Re-run your closed findings on a schedule, because a closed finding is a claim with an expiry date. An audit you run once is a snapshot. An audit on a schedule is a property, which is the half we had been missing. A check that fails out loud beats both, because it does not wait for you to look.

If your team’s harness has been growing for a year and output quality has drifted without anyone being able to say why, that is the work we do: map every control, measure what it actually costs, and rebuild what does not earn its place. We ship the checks that keep a finding closed, not only the finding. We publish the numbers we find, including the ones that make us look bad, which is the standard we held ourselves to here.

References#

[1] N. B. Jones, “Fable 5 And GPT-5.6 Don’t Need Better Prompts. They Need A Clean Setup.,” AI News & Strategy Daily, YouTube, Jul 2026. https://www.youtube.com/watch?v=PDJfciNhyHU

[2] Anthropic, “How Claude Code uses prompt caching,” Claude Code Documentation, 2026. https://code.claude.com/docs/en/prompt-caching

[3] G. Dotzlaw, K. Dotzlaw, and R. Dotzlaw, “Context Engineering for Claude 5: The Six New Rules (and How to Find the Rules You Can Delete),” 2026. /insights/claude-code-19-context-engineering-claude-5-rules/

[4] G. Dotzlaw, K. Dotzlaw, and R. Dotzlaw, “The Context Engineering Stack: Compression, Retrieval, and Decision Memory,” 2026. /insights/ai-16-context-engineering-stack/

[5] G. Tan, “Every company should have a Brain,” AI Engineer, YouTube, 2026. https://www.youtube.com/watch?v=eBUyTS7SzV4

[6] G. Tan, “Do it. Skillify it. Add to cron.,” X, 2026. https://x.com/garrytan/status/2058871289545466300

[7] B. Cherny, “A weird experiment I’ve been trying the last few weeks,” LinkedIn, Aug 2026. https://www.linkedin.com/posts/bcherny_a-weird-experiment-ive-been-trying-the-last-ugcPost-7493781300970147842-u11d/

[8] Y Combinator, “Boris Cherny: We Cut 80% of Claude Code’s Prompt,” YouTube, Jul 2026. https://www.youtube.com/watch?v=qyPCVqFUyDo

[9] G. Dotzlaw, K. Dotzlaw, and R. Dotzlaw, “The Orchestration Tax: Why Loop Engineering Has a Human Ceiling, Not a Token One,” 2026. /insights/claude-code-17-orchestration-tax/