The Unmetered Cost of Agent Skills

by | Jul 28, 2026 | AI, Blogs

Security dominates the conversation about agent skills, and it should. A skill is a set of instructions your agent follows; an attacker who writes those instructions controls your agent, and the published research on malicious skills deserves the attention it gets. I come back to it below. But there is a second exposure I have not seen anyone write about, and for most organizations it will show up first: skills carry a continuous cost that no invoice itemizes, and the format’s design guarantees that this cost grows with adoption.

How skills spend money

A skill is a folder holding a SKILL.md file: plain-English instructions that change how an agent performs a task, optionally bundled with scripts and reference material. The format is an open standard, released by Anthropic and now read by Claude Code, GitHub Copilot, VS Code, Cursor, Gemini CLI, OpenAI Codex, and dozens of other tools.

The design decision that matters here is called progressive disclosure. An agent cannot afford to load every skill in full, so at the start of each session it loads only the name and description of every skill available to it, enough for the model to know what it can reach for. When a task matches, the full instruction text loads into the context window, and in Claude Code it then stays in context for the rest of the session. The documentation is candid about the consequence: once a skill loads, “every line is a recurring token cost.”

Figure 1. One session under progressive disclosure: every available skill pays its description at session start; an invoked skill’s full text stays in context to session end.

At organizational scale, that design makes every shared skill a small recurring tax on every session it can reach. One skill is negligible. A few hundred are a budget line nobody approved.

For one person with a handful of skills, none of this matters. Description overhead runs about a hundred tokens per skill, the full text is only paid for when a skill is used, and the design is sensible; I would not change it.

The multiplication is what deserves attention. Descriptions load per session, and an organization runs a lot of sessions: every seat, several times a day, every working day. Every organization-wide skill adds its description cost to every one of those sessions whether anyone invokes it or not, and a verbose skill that does get invoked costs much more. None of it appears as a line item. It is folded into model spend, where it looks identical to productive work.

Run the arithmetic once to see the scale. Four hundred seats, a dozen sessions a day each, 150 shared skills at roughly a hundred description tokens apiece: 72 million description tokens every working day before a single skill is invoked. The numbers are placeholders; the multiplication is the point.

Figure 2. The article’s illustrative arithmetic. Description overhead only, before any skill is invoked.

The vendors can see the pressure; Claude Code caps each skill’s context listing at 1,536 characters, in the documentation’s words, “to reduce context usage”. A cap limits the worst case per skill, and the per-skill case was never the problem. The problem is how many skills accumulate and how long they stay.

Nothing stands between a skill and your fleet

They accumulate because nothing slows them down. The specification requires two fields, a name and a description. License and compatibility notes are optional. There is no owner field. There is no version, no dependency mechanism, no deprecation state. A skill cannot declare that it replaces another skill, so consolidation is manual work nobody is assigned to, and duplication is the format’s natural condition. Publishing takes a git commit, and reach compounds across tools: skills committed to a repository’s .claude/skills directory are picked up by GitHub Copilot automatically, so a skill written for one assistant becomes context load on every assistant that reads the same path. At no point in that pipeline does anything ask who approved the skill, who maintains it, whether it works, or when it should come out.

The spec’s thinness is deliberate and given the startup loading it is correct; every mandatory field would cost tokens in every session everywhere. The consequence is simply that governance has to live above the specification, and today, for most organizations, nothing lives there.

Enterprise IT has seen this pattern before. SharePoint sites, internal wikis, Jenkins jobs, GitHub Actions, Terraform modules: artifacts that are cheap to create and free to publish accumulate until nobody can say which ones are canonical, which are abandoned, and which contradict each other. New hires inherit the pile without a map. Skills are early on the same curve, with one property the earlier artifacts lacked: every item in the pile bills tokens.

The security exposure, stated plainly

The most useful measurement comes from an empirical study of 632 confirmed vulnerabilities in public skills: 84.2 percent sat in the SKILL.md prose, against 8.5 percent in bundled code. Organizational reflexes point the other way. Code review, static analysis, and dependency scanning inspect the executable minority, while the English majority reads as documentation and passes.

Figure 3. Where 632 confirmed vulnerabilities in public skills sat (the arXiv study linked above). Review tooling inspects the code minority; the prose majority passes as documentation.

Scanning products exist, and the published evasion record should set expectations. Trail of Bits defeated several production scanners with techniques they built in under an hour, one as simple as padding the file with roughly 100,000 newlines to push the payload outside the region the scanner inspected. In another documented case, a research team’s skill passed every scanner it faced, reached roughly 26,000 agents through a marketplace, then changed its behavior after approval; its payload lived at an external URL the authors could edit at will.

My read of that record: run the scanners, use them to prioritize human review, and never accept a clean scan as admission. Two human reviews matter, because the artifact has two halves: a security engineer for anything executable, and a domain reviewer for the instructions, read as carefully as code. Two controls follow from the mutation case. A skill that pulls instructions from a URL you do not control cannot be reviewed at all, whatever a scan says. And a version bump is a new review, because the approval attached to content that no longer exists.

The skills that are merely useless

Between the malicious minority and the skills that earn their keep sits a population that gets almost no attention. SkillsBench, the first substantial benchmark of whether skills improve task performance, found that curated skills raised average pass rates by 16.2 percentage points, which is a strong result and the reason skills are worth an organization’s investment at all. The same measurement found 16 of 84 tasks performing worse with a curated skill than without it, and skills the models generated for themselves providing no benefit on average.

Figure 4. SkillsBench’s 84 tasks (source linked above). The average is strong; the 16 regressions are invisible inside it.

A skill with negative lift is not a neutral failure. It pays the description overhead in every session it reaches, degrades tasks the model previously handled, and sits in the catalog indefinitely, because nothing measures it and nothing retires it. Every control shipped today checks whether a skill is safe, who published it, or what version it carries.

Whether it helps is not checked anywhere.

What I would build

We reviewed the enterprise skills tooling market this month, and the finding, current as of July 2026, is that this problem cannot be procured. The review was Covenant Technology Partners’ work. Neither Anthropic nor Microsoft governs the skill artifact itself; each governs its own surrounding layer, distribution channels on one side and agent identities on the other, and we could not verify any commercial product that fills the gap at enterprise scale. What fills it is modest and buildable:

  • An inventory. Every shared skill in one catalog with a named owner, a person rather than a team. This comes first because nothing else can be enforced, measured, or billed against a population you cannot enumerate.
  • Review at the publish boundary. Personal experimentation stays free; the moment a skill is shared with a team or the organization, the pull request that shares it triggers the two reviews described above. The gate rides on an event that already happens, so nobody has to remember a policy.
  • A small evaluation per shared skill. A handful of representative tasks, run before promotion and rerun occasionally with the skill disabled. Promote on measured improvement, retire on measured harm, and retire skills the model has outgrown, since they have become pure overhead.
  • Usage logging. Vendor telemetry will not tell you which skills earn their context; application-level logging can, and it is the difference between pruning by evidence and pruning by argument.
  • A rehearsed removal. Deleting a skill from the catalog does not remove installed copies. Whatever your path to a clean fleet is, run it once before you need it, and time it.

If you run agent tooling at any real scale, you already have a skills library, whether or not anyone calls it that. People are committing skills to repositories today, the tools are loading them today, and the meter is running today. Skills have quietly become enterprise infrastructure, and infrastructure needs what infrastructure has always needed: an inventory, owners, a lifecycle, and cost controls. The only question is whether anyone can enumerate what you are paying for. Start there.

Where Covenant fits

This gap has a familiar shape: a technology spreading faster than the operational controls around it. That is the kind of problem Covenant Technology Partners helps clients work through, in emerging areas of AI where no precedented solution exists to buy and the exposure is both financial and business risk. Skills governance is a live case: the cost is unmetered, nothing on the market fills the gap, and the controls above are modest and buildable. If the open question in your organization is what you are paying for and who approved it, we can help you answer it.