A new rule needs a failing test first. If you want to add or change a rule in any SKILL.md or domain adapter, bring the trap scenario where the current version fails and your version passes. A rule that sounds wise but does not move a measured number does not ship.
Nulls get reported. If your experiment shows no effect, that is welcome in the results log too. A log that only contains wins is not worth trusting.
- Create
eval/scenarios/<name>/with fixture files. A good trap makes the plausible action the wrong one, and its ground truth is deterministic (checkable by diff, execution, or a source file). - Write
GROUND-TRUTH.md: the task prompt, the trap, scoring caps, ideal behavior. - Do not name the evidence in the task prompt (that pre-solves the scenario).
- Run it A/B (with and without the skill) using
eval/workflow.py, 2+ seeds per cell. - PR the fixture, sanitized judge outputs, and a RESULTS.md entry.
An adapter earns a file only if its four nouns genuinely differ from existing adapters: what counts as evidence, who the authority is, what verification means, and what the frauds are. Follow TEMPLATE.md. Bring the discriminating scenario. Sectors that fold into existing adapters get a routing line, not a file. Medical/clinical stays excluded.
- No em dashes or en dashes. Use commas, colons, parentheses, or two sentences. CI enforces this.
- Skills stay lean. Depth goes in
references/, loaded on demand. - Run
python .github/checks.pybefore pushing.
- Frontmatter:
name,description(trigger-focused, under 1024 chars),version,author,license,metadata.hermes.tags,metadata.hermes.related_skills - Directory:
skills/<category>/<skill-name>/SKILL.md - Supporting files:
references/,templates/,scripts/subdirectories - Total file under 100,000 chars (aim for 8-15k)