A skill library is the one part of a harness that keeps getting better the more you feed it, right up until it doesn’t. Somewhere past the point where you can hold the whole list in your head, the agent starts picking a skill that sounds like the task while missing the one that makes the task possible, or loading two skills that quietly work against each other. The instinct is to blame the model. One 2026 preprint argues that the problem sits upstream of the model, in the retrieval step, and that the retrieval step has been solving the wrong kind of problem the whole time.
One thing to clear up first, since it sits in our own title. The number 20 is ours. No paper in this literature names a skill count at which selection breaks, and we are not going to invent one. SkillDAG, the preprint this article leans on hardest, runs a 37-skill library on one benchmark and libraries of 200, 500, 1,000, and 2,000 on another [1]. Anthropic publishes a character budget for its skill listing and never a skill count [2]. The symptom showed up in our own harness at roughly 20 skills, so read it as a marker for “past the handful you can keep in your head,” not a measured threshold.
Every arXiv paper cited below is an unrefereed preprint. None shows a venue or an acceptance, and one of them, SkillGraph [3], is explicitly marked “Under Review” on arXiv. Treat what follows as a research direction with one concrete product symptom attached, not settled practice.
Figure 1 - Similarity versus structure: Flat retrieval embeds the task, scores every skill against it, and concatenates the top matches. Every relationship between those skills, prerequisite, conflict, specialization, redundancy, collapses onto that one number. The proposed alternative keeps the relationships as typed edges and lets the agent walk them.
Similarity is the wrong question
SkillDAG states the case in a single sentence [1]:
“As LLM agents adopt large skill libraries, selecting the right subset becomes a structural problem rather than a similarity-matching one: skills depend on, conflict with, specialize, or duplicate one another, a structure invisible to both full enumeration and embedding similarity.”
Four concrete consequences follow, each recognizable to anyone who has watched a harness misfire [1]:
- Silent prerequisite omission. “the top match names the task while a functionally necessary prerequisite (a parser, converter, or setup utility) is silently omitted” [1].
- Interference on co-selection. Two skills that each work fine alone degrade each other when loaded together.
- Near-duplicate context waste. Redundant skills spend budget to say the same thing twice.
- Missed joint capability. “pairs that jointly enable a capability neither provides alone” [1].
Graph-of-Skills, the April 2026 preprint SkillDAG benchmarks itself against, names the first of those the prerequisite gap: “semantic retrieval surfaces topically relevant skills but misses their prerequisite chain of upstream and downstream skills, creating a prerequisite gap that leaves the retrieved bundle execution-incomplete” [4].
The word to sit with is execution-incomplete. The bundle is not off-topic. It is topically perfect and functionally short a piece, the hardest failure to catch from a log, since everything the agent loaded looks correct.
The number that reframes the topic
On SkillsBench at the 1,000-skill scale, SkillDAG’s Table 1 reproduces, from Graph-of-Skills, flat vector retrieval scoring below dumping the entire library into the prompt with no retrieval at all: 10.4 reward against 17.2 on MiniMax-M2.7, and 21.5 against 27.4 on gpt-5.2-codex [1] [4]. Flat vector retrieval’s intrinsic quality on the same setup is a Ret@1 of 3.6 [1]. Retrieval is not adding less value than a graph would. It is subtracting value relative to doing nothing clever at all.
That deserves a caveat and a complication.
The caveat: those baseline rows were not rerun by SkillDAG’s authors. The Vanilla, Vector, and Graph-of-Skills rows “are quoted from li2026gos Table 1 and were not rerun locally” and are “not paired seed-matched reruns of the baselines” [1]. Graph-of-Skills is SkillDAG’s direct baseline rather than an independent party confirming its result, so every headline delta in that table compares against numbers the authors copied. Both of the rows being compared here are Graph-of-Skills’ own measurements, and Graph-of-Skills is a paper arguing that you should replace flat retrieval with a graph. No independent group has reproduced the negative result.
The complication matters more: on the smaller ALFWorld library of 37 skills, vector retrieval does help, scoring 50.7 against the no-retrieval 47.1 on MiniMax-M2.7 [1]. Same table, same paper, opposite sign. This is a scaling failure, not a blanket indictment of embeddings. Vector retrieval helped on ALFWorld’s 37-skill library and hurt on SkillsBench’s 1,000-skill one. Read that as a scaling signal, not a scaling law.
Figure 2 - The sign flips with scale: At a small library size, flat similarity retrieval beats loading everything. At a much larger library size on a different benchmark, it falls below the no-retrieval baseline. Library size is not the only thing that differs between those two rows, so treat the shape as suggestive rather than measured.
KEY INSIGHT: In SkillDAG’s Table 1, retrieval that helps at 37 skills hurts at 1,000. Measure skill selection against the dumbest baseline, loading everything, before assuming a retriever earns its place.
The same wall, already shipping
The most useful thing in this corpus is not in a paper. It is in Claude Code’s documentation, arrived at from the opposite direction.
Claude Code preloads a listing of every skill’s name and description so the model knows what exists. That listing has a hard budget, and Anthropic documents what happens when it overflows [2]:
“The listing always contains every skill name, but if you have many skills, Claude Code shortens descriptions to fit the listing’s character budget, which can strip the keywords Claude needs to match your request. The budget scales at 1% of the model’s context window. When the listing overflows, Claude Code drops descriptions starting with the skills you invoke least, so the skills you use most keep their full text.”
Read the eviction rule again. The order is invocation frequency, not relevance to the task in front of you. A skill you run daily keeps its full description. A skill you have run twice this year loses its description first, and a name-only entry gives the model almost nothing to match against.
Now put that next to the prerequisite gap. The skill most likely to be today’s missing prerequisite, the converter or setup utility you touch once a quarter, is exactly the entry the budget strips first. That is a context-budget heuristic producing the same kind of harm a retrieval score produces, in a product people use right now. The mechanisms differ: Graph-of-Skills’ prerequisite gap is about a retrieved bundle arriving execution-incomplete, while listing eviction is about a skill becoming hard to match at all once its description is gone. Anthropic’s own wording is that truncation “can strip the keywords Claude needs to match your request” [2]. Whether it does so often enough to matter in practice is not something anyone has published.
Figure 3 - Eviction by habit, not by need: When the skill listing exceeds its budget, the entries that lose their descriptions are the ones invoked least often. Frequency of past use is a reasonable proxy for future use and a poor proxy for today’s prerequisite, which is precisely the skill you reach for rarely.
Two absences are worth naming just as carefully, since they are the whole argument for a typed graph. Anthropic documents no dependency ordering between skills, and no semantic conflict detection between differently-named skills [2]. There is a precedence rule, enterprise over personal over project [2], but that resolves name collisions by filesystem shadowing. It is not semantic conflict resolution, and two differently-named skills that fight each other at runtime stay invisible to it.
The practical knobs are all mitigations rather than fixes. The per-entry cap is 1,536 characters covering the combined description and when_to_use text, which is why Anthropic’s advice is to put the key use case first [2]. The budget fraction is raisable via skillListingBudgetFraction, low-value entries collapse to "name-only" through skillOverrides, and /doctor estimates the listing’s context cost and its biggest contributors [2]. If you have never checked that number for your own harness, that is the highest-value thing you can do after reading this.
Figure 4 - Write the description for the retriever: A description is truncated from the tail. Front-loading the trigger case means the part that survives truncation is the part that has to match. Prose that builds to its point is prose that loses its point to a character cap.
KEY INSIGHT: Skill descriptions are retrieval surface, not documentation. Write the trigger case in the first sentence, because the tail is what a budget deletes and the skills you use least are the first to lose theirs.
This is the discipline question from Skills, Slash Commands, and Harnesses: A Discipline Hierarchy [5] seen from the retrieval side. A skill that is never found is indistinguishable from a skill that was never written.
Five edge types, enumerated
SkillDAG’s proposal is to type the relationships instead of discarding them. Its typing function assigns every edge exactly one of five labels, and the list is worth reproducing exactly rather than rounding off [1]:
| Edge type | Paper’s definition | What it does at retrieval time |
|---|---|---|
depends_on | ”marks u as requiring the prerequisite v” | Selecting u pulls v in as a prerequisite |
specializes | ”marks u as the narrower variant to prefer over v” | Prefer the specialized skill over the general one |
composes_with | ”marks synergistic co-use” | Surfaces composition opportunities |
similar_to | ”marks functional redundancy (pick one)“ | De-duplicates, select one and not both |
conflicts_with | ”marks pairs that should not be co-selected” | Prunes, selecting u excludes v |
Two structural rules fall out of that typing, both more interesting than the list itself.
First, conflicts prune and never navigate. The paper excludes conflicts_with from traversal deliberately, since “navigating a conflict edge would actively undermine retrieval by surfacing the very skills the LLM has been told to avoid” [1]. Four types are walkable. The fifth exists purely to exclude.
Second, the DAG in the name covers only part of the graph. depends_on and specializes form “the directed backbone whose acyclicity gives SkillDAG its name” [1], while composes_with and similar_to are symmetric and sit outside the acyclicity constraint. Calling the whole structure a DAG is a simplification the paper does not make.
Figure 5 - Four edges you walk, one you obey: Dependency, specialization, composition, and redundancy edges are traversed to expand a candidate set. The conflict edge is never traversed, only used to exclude, since walking it would surface exactly the skills the agent was told to keep apart.
One warning before anyone standardizes this. Three papers use three incompatible edge vocabularies. SkillDAG uses the five above [1]. SkillGraph uses prerequisite, enhancement, and co-occurrence [3]. GraSP models precondition-effect relations [6]. There is no agreed taxonomy, and flattening them into one would misrepresent all three.
Handing the graph to the agent, not to a ranker
The design inversion matters more than the edge list. SkillDAG does not feed the graph to a ranking policy that returns a bundle. It exposes the graph to the model as four callable operations, search, show, propose-edge, and edit-edge, and its own Figure 1 caption states the thesis without hedging: “The LLM, not a graph-ranking policy, decides what to load” [1].
A single search returns three separate fields rather than one fused score [1]:
- matches, the top-K by embedding cosine against the query. Similarity survives, demoted from answer to input.
- neighbors, a bounded breadth-first walk over the four traversable edge types, each neighbor carrying its distance from a match, its predecessor, and the edge type that got it there.
- conflicts, the one-hop
conflicts_withedges touching any match, with no transitive expansion.
Keeping the fields separate is the point. The agent can “drop an entire field under a tight context budget,” “schedule search calls on its own initiative,” and “leave a reasoning trace documenting why each skill was selected or excluded” [1]. None of that survives a fused relevance score, which returns a ranked list and no account of itself.
Skill bodies load only when the agent calls show. That is progressive disclosure, the loading discipline Anthropic describes for Agent Skills generally [7], applied to a graph instead of a folder.
Figure 6 - Three fields, not one score: A search returns topical matches, typed graph neighbors, and conflict warnings as separate channels. The agent decides what to keep, what to drop under budget pressure, and what to explain, none of which is possible once the three collapse into a single ranked list.
KEY INSIGHT: A retriever that returns one ranked list has already made the decision. One that returns separated evidence lets the agent make it, and lets you read why afterward.
Cold start: one embedding per skill is the wrong shape
The construction step holds the most portable idea in the paper, and it has nothing to do with graphs.
SkillDAG embeds each skill twice. Once for what the skill does, from its identifier, description, and a body preview. Once for what the skill requires, by having a model imagine two or three invoking tasks and summarize the shared prerequisites in one sentence, which is then embedded [1]. A candidate pair survives if either ranking clears an adaptive threshold, after which a classifier assigns one of the four static types or none.
The worked example is the entire argument in miniature: “A cooling skill and a pickup skill have disjoint self-descriptions, yet the cooling skill requires ‘object in hand’ produced by the pickup skill. Self-description similarity alone systematically misses such cross-functional pairs” [1].
That generalizes past skill libraries. Storing one vector per item asserts the item has exactly one useful similarity axis. A skill has two, what it is and what it needs, and they point at different neighbors. The paper reports no ablation isolating the two-view embedding’s contribution to its headline numbers, so this is a design argument rather than a measured one. It is persuasive because the failure it describes is easy to check by hand, not because a table separates it out.
Figure 7 - One vector, one axis: Embedding a skill by what it does finds skills that sound like it. Embedding the same skill by what it requires finds the upstream skills that make it runnable. Those are different neighborhoods, and a single vector per skill can only represent one of them.
Propose-then-commit, and the honest limit of the evidence
Conflict edges cannot be built at construction time at all, and the reasoning is clean: “a conflict is, by definition, a pair whose co-use predictably degrades success, and that evidence simply does not exist until execution is observed. No amount of static reading recovers it” [1].
So the graph evolves at runtime through two separated calls. propose-edge previews the change against existing edges and history for the pair. edit-edge commits, “with a natural-language reason and the supporting execution trace” [1]. The split exists so committing is “always a deliberate act rather than a side effect of execution” [1]. Three invariants bound the damage: acyclicity on the backbone, non-contradiction so a pair cannot carry a conflict edge alongside a positive one, and reversibility through an append-only history [1].
Replaying 316 queries after 27 committed edits, the edited graph lifted mean ground-truth skills retrieved per query from 1.915 to 1.984, and “zero queries lose any ground-truth retrieval” [1]. That property is set-monotonicity on recall, and the authors are explicit that it is a recall property rather than a token-savings one. An accepted edge can only enlarge the retrieved set, which is exactly why it cannot double as a context-budget win.
Figure 8 - Deliberate by construction: Proposing an edge previews it against existing history. Committing requires a stated reason and an execution trace, and passes three invariants before it lands. The separation exists so the graph never changes as a side effect of a task completing.
The advantage is largest in the middle
The result that keeps this from being a sales pitch sits in the same table as the headline numbers. On the strongest backbone tested, gpt-5.2-codex, SkillDAG ties Graph-of-Skills on ALFWorld at 93.6 [1]. Not beats. Ties. The paper’s own explanation is that “the routing aid signature is large where the backbone cannot reason past a poor bundle and vanishes once it can” [1]. Structural retrieval is worth the most to a model that cannot recover from a bad skill bundle on its own, and worth progressively less as the model gets better at recovering.
A separate May 2026 preprint finds the same shape from a different direction. Splitting self-evolution into the capability to write harness updates and the capability to benefit from them, a six-institution group across Penn State, UC Santa Cruz, Amazon, Emory, UIUC, and Northeastern reports that “harness-updating is flat in base capability,” to the point that “even Qwen3.5-9B’s updates yield gains comparable to those of Claude Opus 4.6,” while “harness-benefit is non-monotonic in base capability: weak-tier models benefit little from updated harnesses, mid-tier models benefit most, and strong-tier models benefit less than mid-tier” [8]. Read the first half carefully. The claim is that the gains produced by the updates are comparable, not that a 9B model and a frontier model are interchangeable as evolvers.
The two results agree on part of a shape without either one establishing all of it. SkillDAG’s two backbones show the falling half: the routing aid is large on the weaker one and gone on the stronger one. The rising half, that weak models benefit least because they fail to invoke the artifact at all, comes from the self-evolution paper alone [8]. Neither result establishes the full curve on its own, and they were measured on different things, so the agreement is suggestive rather than a replication. The authors’ own recommendation follows from their half of it: invest capability budget “in the task-solving agent rather than the evolver” [8].
Figure 9 - Structure helps most in the middle: The full curve here is the self-evolution paper’s finding across capability tiers, where weak models fail to activate harness artifacts, mid-tier models gain the most, and the strongest close the gap on their own. SkillDAG’s two backbones sit on the falling half of it only. The practical reading either way is that a structural advantage measured on one backbone should never be assumed to transfer to a better one.
Eno Reyes, co-founder and CTO of Factory, offers a product-side reading of why: “clearly the harness informs the model in a very interesting way,” with post-training building “a gravity towards a workflow that solves the problem very effectively” [9]. On that account a harness elicits workflows a model already has, which pairs with the pruning argument in The Harness Evolution Principle [10].
What any of this changes for a growing library today
Nothing here ships as a drop-in retriever for a personal skill library. SkillDAG is a research repository [11], the benchmark runs are its authors’ own, and the baselines it beats were quoted rather than rerun. Anyone selling a skill graph to install this afternoon is ahead of the evidence.
Four things transfer, and none requires new tooling.
Treat the library as a graph you already have and have never written down. The prerequisite chains, the conflicting pairs, the general skill a narrower one should override, all of it exists in your harness now. Nothing indexes it, so the knowledge lives in whoever wrote the skills.
Prefer a few focused skills over one exhaustive one. This is the one finding supported by groups other than SkillDAG’s authors. The two results measure different units, and it is worth keeping them apart. SkillsBench is about composition inside one skill: curated skills “raise the average pass rate from 33.9% to 50.5% (+16.6 percentage points)” and “Focused Skills with at most three modules outperform larger or exhaustive bundles” [12]. GraSP is about how many skills you hand the agent, and reports from the orchestration side that “providing agents with more skills does not monotonically improve performance” and that “focused sets of 2-3 skills outperform comprehensive documentation, and excessive skills actually hurt” [6]. Its conclusion is the line to remember: “The bottleneck has shifted from skill availability to skill orchestration” [6].
Write descriptions for the retrieval step, not for a human reader. Key use case first, trigger phrasing early, assuming everything after the first sentence may be deleted by a budget you never see.
Expect the conflict relations to cost something to learn. Prerequisites can be read off skill files with effort. Conflicts cannot. They appear only when two skills run together and the output gets worse, so the only way to find one is to notice a bad run and write down why, somewhere your future harness can read it.
Figure 10 - Four practices that need no new tooling: Write down the relationships that already exist, keep skills focused rather than exhaustive, front-load descriptions for the retriever, and capture conflicts when a run degrades. None of these require a graph database, and all of them survive whatever retrieval the vendors ship next.
Conclusion
The most useful sentence in this literature is that skill selection is a structural problem rather than a similarity-matching one. Everything else follows. If prerequisites, conflicts, specializations, and duplicates are real relations between skills, compressing them into one cosine score discards the information that decides whether a bundle runs, and the evidence that this costs something is a benchmark where retrieval scores below no retrieval at all.
The proposed fix, a typed directed graph exposed to the agent as a callable interface rather than buried in a ranker, has real numbers behind it and real limits around it. The advantage was measured against baselines that were quoted rather than reproduced, it disappears on the strongest backbone tested, and the self-evolution property is about recall rather than tokens. That makes it something to reason with rather than something to adopt.
What is not speculative is the version already shipping. A skill listing on a 1% context budget that drops descriptions starting with the skills you invoke least is a documented, in-production mechanism by which a growing harness can lose the very entry it needs, and Anthropic documents the risk in its own words. Checking what your listing costs, front-loading your descriptions, and keeping skills focused are worth doing this week, whatever the retrieval layer looks like a year from now.
References
[1] T. Bai, Z. Wan, P. Zhou, X. Yu, Y. You, and I. W. Tsang, “SkillDAG: Self-Evolving Typed Skill Graphs for LLM Skill Selection at Scale,” arXiv:2606.03056, Jun 2026. https://arxiv.org/abs/2606.03056
[2] Anthropic, “Extend Claude with skills,” Claude Code Documentation, 2026. https://code.claude.com/docs/en/skills
[3] X. Li et al., “SkillGraph: Skill-Augmented Reinforcement Learning for Agents via Evolving Skill Graphs,” arXiv:2605.12039, May 2026. https://arxiv.org/abs/2605.12039
[4] D. Liu, Z. Li, et al., “Graph-of-Skills: Dependency-Aware Structural Retrieval for Massive Agent Skills,” arXiv:2604.05333, Apr 2026. https://arxiv.org/abs/2604.05333
[5] G. Dotzlaw, K. Dotzlaw, and R. Dotzlaw, “Skills, Slash Commands, and Harnesses: A Discipline Hierarchy,” 2026. /insights/claude-code-06-skills-slash-harnesses/
[6] T. Xia et al., “GraSP: Graph-Structured Skill Compositions for LLM Agents,” arXiv:2604.17870, Apr 2026. https://arxiv.org/abs/2604.17870
[7] B. Zhang, K. Lazuka, and M. Murag, “Equipping agents for the real world with Agent Skills,” Anthropic Engineering Blog, Oct 2025. https://www.anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills
[8] M. Lin, J. Wu, et al., “Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents,” arXiv:2605.30621, May 2026. https://arxiv.org/abs/2605.30621
[9] E. Reyes, “The best AI agents cost less than you think,” LangChain, Jul 2026. https://www.youtube.com/watch?v=HbUznYhKFOc
[10] G. Dotzlaw, K. Dotzlaw, and R. Dotzlaw, “The Harness Evolution Principle: Why Mature Harnesses Look Like Pruning,” 2026. /insights/claude-code-05-harness-evolution/
[11] Ericbai06, “Ericbai06/SkillDAG,” GitHub, 2026. https://github.com/Ericbai06/SkillDAG
[12] X. Li et al., “SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks,” arXiv:2602.12670, Feb 2026. https://arxiv.org/abs/2602.12670
Building production AI, or modernizing a legacy system?
That is the kind of work we do at Dotzlaw Consulting. Book a free 20-minute intro call and tell us what you are trying to build, or what is slowing you down.