Rename the agent guidance to AGENTS.md and explain the Babel domain it assumes

[gaurav]

CLAUDE.md had grown to 396 lines by accretion — each hard-won rule from a review round appended wherever it fit. It was accurate about this repository's internals and silent about the system those internals exist to read: it never said what Babel is, what a clique or a concord is, or why NodeNorm's version matters. An agent arriving at the Concord.parquet query code had no model of the data.

This renames the file to AGENTS.md, gives it that missing context, moves the deep caching invariants to docs/, and removes what was being said twice. Documentation only — no change to src/.

What's here

Renamed to AGENTS.md, the filename coding agents other than Claude Code look for; the guidance was never Claude-specific. CLAUDE.md remains as a four-line pointer, so Claude Code still finds it. Four test docstrings that referenced the old name are updated. The rename is the first commit, so the substantive edits that follow read as edits rather than as a delete plus an add.

A Domain context section covering only the terms this codebase actually uses — clique, preferred identifier, concord, conflation, Biolink type — and NodeNorm's relationship to a specific Babel release. It is a summary with links rather than copied upstream prose, so it cannot drift from upstream unnoticed.

The part that earns the most space is the boundary between concords and cliques. Babel's own AGENTS.md says to answer clique-membership questions from a finished build "never from the concords that fed it" — and concords are exactly what this tool reads. That is not a defect, it is the purpose: xrefs reports the evidence Babel read, which is what you want when a merge looks wrong, while NodeNorm reports the verdict. Written down explicitly so that nobody later "improves" xrefs into a clique oracle.

The caching invariants moved to docs/Downloading.md — near-verbatim, because the wording is hard-won and each rule records a specific failure that actually happened. AGENTS.md keeps a four-sentence summary and a pointer. That is ~90 lines an agent needs rarely, which had been sitting above the orientation it needs every session.

Deduplication. The DuckDB spill directory was described in three places, the caching model in three, and the never-commit-the-internal-URL rule in three. Each is now stated once, where it belongs. ## Important Notes is deleted outright — all three of its bullets restated earlier sections. The command examples now point at README.md, which documents every flag and is the user-facing reference.

Four factual corrections:

• MissingBabelFileError is raised on any 404, not only for duckdb/ paths, as the file claimed in two places. downloader.py:546 has no path condition.
• CI budgeting pointed only at CI cannot run the Parquet integration tests #18; CI never exercises the integration tests, because it has no Babel release that publishes the Parquet files #28 covers the same ground more precisely.
• "recreated as Add babel-explorer: a CLI for querying Babel cross-references via DuckDB and NodeNorm #20-Add Autocomplete playground tool to Astro/Vue frontend #24" predated Add babel-explorer: a CLI for querying Babel cross-references via DuckDB and NodeNorm #20 merging.
• README.md's configuration table was missing BABEL_ALLOW_VERSION_MISMATCH, which env.default has carried since Skip absent test dependencies, and stop unit runs deleting the integration cache #27.

What upstream does not document

Worth recording, because it shaped what could honestly be written. Neither Babel's README.md nor its releases/ARTIFACTS.md describes Concord.parquet, Identifiers.parquet or Metadata.parquet. conflate / drug_chemical_conflate and the /status babel_version field are absent from NodeNormalization's README.

So the file schema in the new section is what this repository's code assumes, verified against real files — and it is labelled that way rather than presented as a published contract. Release naming, latest and VERSION.txt are undocumented upstream too. Filed upstream as NCATSTranslator/Babel#1077, which links back to the Domain context section as the shortest statement of what is missing.

What it deliberately does not do

• Document search-xrefs — that is Add search-xrefs command with OLS4 and MyChem.info providers #21, not on main.
• Rewrite the version-skew wording — Treat a Babel/NodeNorm version mismatch as a warning, not a failure #30 will change that behaviour; editing the prose now means editing it twice.
• Split Architecture or Testing into docs/ — those are context an agent needs often, and spreading them costs more than the length saves.

Verification

272 unit tests pass and ruff is clean (no source changes, so both should be unchanged). Every relative link and intra-file anchor in AGENTS.md, docs/Downloading.md and CLAUDE.md resolves, no .py file still references CLAUDE.md, and the symbols the reworked text names were re-checked against the code.

Nothing is blocking this merge.

One thing to judge: AGENTS.md is 370 lines, down from 396, with a further 84 in docs/. Total documentation went up, because the Domain context section came out at 61 lines rather than the ~25 originally scoped. The extra went on the concord/clique boundary above. If that reads as too much for an agent file, the obvious trim is ## Testing (73 lines, now the largest section).