An Overview Is Not an Instruction

Somebody finally measured

If your repository has an AGENTS.md, or someone on your team is about to add one, this piece is about what belongs in it. Two research groups have now measured what these files actually do to coding agents, and the results argue for a much shorter file than the one most teams are writing.

AGENTS.md is a Markdown file at the root of a repository that coding agents read before they work. OpenAI published the convention in August 2025 and donated it to the Agentic AI Foundation last December, alongside MCP and three other projects. Somewhere between six hundred thousand and eight hundred thousand of these files now sit in public repositories. Almost nobody has checked whether they work.

A group at ETH Zurich did, and their paper is worth reading in full if this file matters to your team. Gloaguen, Mündler and colleagues built a benchmark from real pull requests in twelve Python repositories that already had developer-written context files, ran four coding agents over the tasks with and without those files, and published the result in February 2026. Success rates did not generally improve. Inference cost rose by more than twenty percent. Neither headline difference reaches statistical significance, which matters for how you read this: the honest claim is that the files did not help, not that they measurably hurt.

The number that should stop you is not the average. It is what happened per model. The same developer-written files made one agent worse by roughly three points and another better by roughly three points. One file, two models, opposite signs. Nobody’s AGENTS.md has been evaluated against the model that will read it next quarter.

Before this becomes an argument for deleting the file, take the other study with it. A second group measured a different axis on 124 pull requests and found median wall-clock runtime down by about twenty-nine percent and output tokens down by about seventeen, with task completion unchanged. Both results can hold at once, and together they say something the marketing does not. An AGENTS.md is an efficiency and consistency tool. It is not a capability tool. Agents do not become more able because you wrote a file. They become more predictable, and they get there faster.

Obeyed, ignored, and expensive

Sit with the ETH result a moment longer, because underneath the headline is the part you can act on. Instructions are followed, and followed hard. When a context file named uv as the package manager, the agents reached for it about a hundred and sixty times more often than when the file was absent. Repository-specific tooling saw roughly two and a half times the use. Telling an agent which command to run works, and it works reliably.

Repository overviews do not. The paper is blunt about it: agents took similar steps to locate relevant files whether or not an architectural summary was provided. The map did not help them navigate, because they were going to open the files anyway.

Now put that next to what people actually write. A survey of 2,303 context files found architecture content in about sixty-eight percent of them, testing in about seventy-six, and security in under fifteen. The most common material in these files is the material the measurement says is inert, and the sections vendors recommend first, the overview and the architecture tour, are the ones the agent skims past on its way to the code.

The sharpest instrument in the ETH paper is an ablation, and it explains why. Strip the README, the docs and the examples out of the repository, and suddenly the context files start helping. Their content was mostly a restatement of documentation the agent could already reach. A line that repeats what the repository already says is not free. It is paid for twice, once in tokens and once in the attention it takes from the lines that matter.

Two tests for every line

So what should go in the file? The cleanest answer I have found comes from Francis Eytan Dortort, whose write-up is the one I would send a colleague before any of the vendor guides. A line earns its place if it does one of two jobs. It resolves a genuine ambiguity, meaning several valid patterns exist and the code does not say which one this project prefers. Or it caches an expensive inference, meaning something the agent could work out, but only after several file reads, some git archaeology, or tracing across modules. Everything else is tokens.

Apply that and the good files stop looking arbitrary. Astral’s uv ships twenty-five lines with no headings at all, a flat list where every line opens with a graded imperative:

- NEVER update all dependencies in the lockfile and ALWAYS use
  `cargo update --precise` to make lockfile changes
- NEVER assume clippy warnings or test failures are pre-existing,
  it is very rare that `main` has warnings
- AVOID shortening variable names, e.g., use `version` instead of `ver`

That file has been committed four times in its life. It was close to right the first time, because every line is a decision the code cannot express. Ghostty’s file is thirty-nine lines and annotates the cost of the obvious command rather than just naming it: run the targeted test with a filter, because the full suite is slow. Kubernetes spends most of its file on generated-code landmines, telling the agent which files never to hand-edit and which command regenerates them. Sentry tells the agent not to run tsc directly and not to run pytest bare, and adds a fact no human contributor would ever need: the agent cannot see the development server’s terminal, so here is the file to tail instead.

Every one of those is a prohibition, a command, or an operational fact that is invisible from the code. None of them is an architecture paragraph. And the failure they prevent is specific: without an explicit rule naming the package manager, practitioners report agents defaulting to the wrong one about half the time.

What the long files cost

Now compare that with the file you actually have. The same survey that counted content types found the median AGENTS.md runs around 335 words. The files practitioners praise sit at or below that: ghostty at 214 words, uv at 279. Famous repositories run several times longer, which is why copying them is bad advice. Apache Airflow’s root file is about 35,800 bytes, which is larger than the 32 KiB budget OpenAI’s Codex applies to concatenated context by default, so some of it would fall off the end before a nested file is even considered.

Length is not just a cap problem. It is an attention problem, and the research on long inputs has a name for it: performance grows less reliable as context grows, not because a limit is hit but because attention spreads. Anthropic’s own guidance warns that a bloated instruction file makes the model ignore the instructions inside it. The irony is exact. The paragraphs you added to be thorough are what push the one line that mattered out of reach.

Which brings up the maintenance finding, and it is the one that should change how you treat the file. Across that corpus, roughly six in ten files were modified in more than one commit, and deletions were negligible. These files only grow. Every rule anyone ever added is still in there, including the ones written to compensate for a model that has since been replaced.

So treat the file as a cache, because that is what it is: a cache of inferences that were expensive when you wrote them. Caches need eviction. Every line encodes an assumption about what the agent cannot work out on its own, and those assumptions expire as models improve. A healthy AGENTS.md gets shorter over the year, not longer. Delete a rule, run the work that rule was protecting, and see whether the agent still gets it right. That is a ten-minute experiment, and it is the only maintenance anyone in this field is doing badly enough to be worth starting.

Not yet a standard

One more thing to know before you build process around this. AGENTS.md is called an open standard, and it is governed by a real foundation, but it does not yet have a specification. The canonical repository is a website plus its own instruction file. There is no schema, no conformance language and no version number, and an open issue asking for a specification document puts it plainly: it is not really a standard if no requirements are defined. That issue has no maintainer reply, and the repository has had no commit since March.

The gap shows up where you would expect. The site says the nearest file wins; Codex concatenates from the root down and lets later files override, which is a different rule with a different failure. Copilot reads the file but ranks its own instructions file above it. Zed reads it seventh in a chain of nine. Gemini’s CLI reads it only if you configure the filename yourself. And this week the largest feature request in Claude Code’s history, asking for native support, was closed as completed without native support ever shipping, with a suggestion to symlink or import instead.

The practical answer is the symlink, and it is what serious repositories already do. Next.js keeps one file and points the other name at it, then writes that fact into the file so nobody edits the wrong one:

ln -s AGENTS.md CLAUDE.md

# or, where symlinks are awkward, make CLAUDE.md a one-line import:
@AGENTS.md

None of that is a reason to skip the file. It is a reason to keep your expectations at the right level. Adopt AGENTS.md because one short file, read by most agents, makes your repository behave consistently for whoever walks into it next. Do not adopt it because a foundation logo implies a specification that has not been written, and do not put anything in it that has to hold, because a rule a model may override is a suggestion, not a control.

If you take one thing back to your own repository, make it this. Open your AGENTS.md and read it line by line, and for each line ask whether the agent would really get this wrong without being told. Some of those lines never earned their place. Others earned it a year ago, against a model that has since been replaced, and nobody went back to check. Delete a few, run the work they were protecting, and see whether anything breaks. Most teams are still adding to these files. The useful work now is taking things out.