Skip to content

Commit f356849

Browse files
roli-lpciclaude
andauthored
audit: DOI-readiness — canon metadata, coherence, honest claims (#1)
Add .zenodo.json + CITATION.cff with ORCID and a tiered keyword set; shrink the About section to a short blurb plus a link to hermes-labs.ai; normalize identity to the hermes-labs-ai org and the Hermes Labs name; remove any claim not backed by an artifact committed in this repo; open the README with a plain entity-definition sentence. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 2474feb commit f356849

5 files changed

Lines changed: 64 additions & 60 deletions

File tree

.zenodo.json

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
{
2+
"title": "intent-verify: a deterministic spec-drift checker for markdown specs and codebases",
3+
"description": "intent-verify is a deterministic spec-drift checker that verifies whether a repo's code still covers the acceptance items stated in a markdown spec, INTENT.md, or handoff doc. It parses acceptance items, scores lexical token overlap against the codebase, and returns verified / partial / missing so teams catch repo intent drift before review, release, or handoff. Deterministic, zero-LLM, no network.",
4+
"upload_type": "software",
5+
"creators": [
6+
{ "name": "Bosch Rodriguez, Rolando", "orcid": "0009-0005-4896-1112", "affiliation": "Hermes Labs" }
7+
],
8+
"keywords": [
9+
"AI reliability",
10+
"LLM reliability",
11+
"spec-drift checker",
12+
"silent failure modes",
13+
"spec drift detection",
14+
"repo intent verification",
15+
"handoff verification",
16+
"zero-LLM"
17+
],
18+
"license": "MIT",
19+
"version": "0.1.0",
20+
"related_identifiers": [
21+
{
22+
"identifier": "10.5281/zenodo.19042469",
23+
"relation": "references",
24+
"scheme": "doi"
25+
}
26+
]
27+
}

CITATION.cff

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
cff-version: 1.2.0
2+
message: "If you use this software, please cite it as below."
3+
title: "intent-verify: a deterministic spec-drift checker for markdown specs and codebases"
4+
abstract: "intent-verify is a deterministic spec-drift checker that verifies whether a repo's code still covers the acceptance items stated in a markdown spec, INTENT.md, or handoff doc. It parses acceptance items, scores lexical token overlap against the codebase, and returns verified / partial / missing so teams catch repo intent drift before review, release, or handoff. Deterministic, zero-LLM, no network."
5+
type: software
6+
authors:
7+
- family-names: "Bosch Rodriguez"
8+
given-names: "Rolando"
9+
orcid: "https://orcid.org/0009-0005-4896-1112"
10+
affiliation: "Hermes Labs"
11+
license: MIT
12+
version: "0.1.0"
13+
date-released: "2026-04-19"
14+
repository-code: "https://github.com/hermes-labs-ai/intent-verify"
15+
references:
16+
- type: article
17+
title: "A Taxonomy of Epistemic Failure Modes in LLMs"
18+
authors:
19+
- family-names: "Bosch Rodriguez"
20+
given-names: "Rolando"
21+
doi: "10.5281/zenodo.19042469"
22+
keywords:
23+
- "AI reliability"
24+
- "LLM reliability"
25+
- "spec-drift checker"
26+
- "silent failure modes"
27+
- "spec drift detection"
28+
- "repo intent verification"
29+
- "handoff verification"
30+
- "zero-LLM"

LICENSE

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
MIT License
22

3-
Copyright (c) 2026 Rolando Bosch
3+
Copyright (c) 2026 Hermes Labs
44

55
Permission is hereby granted, free of charge, to any person obtaining a copy
66
of this software and associated documentation files (the "Software"), to deal

README.md

Lines changed: 5 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,6 @@
1-
# intent-verify: repo intent verification and spec drift checks
1+
# intent-verify is a deterministic spec-drift checker
2+
3+
intent-verify is a deterministic spec-drift checker that verifies whether a repo's code still matches a markdown spec via lexical token overlap — repo intent verification and spec-drift checks. Zero-LLM.
24

35
Find spec drift fast when your repo has an `INTENT.md`, `SPEC.md`, or handoff doc but nobody knows if the code still matches it.
46

@@ -84,7 +86,7 @@ intent-verify: INTENT.md vs . (12 files)
8486
[OK 100%] uploads PDF invoices
8587
[PART 50%] retries provider timeout
8688
[LOW 20%] writes audit log for rejected invoices
87-
intent-verify: MISSING — 1/3 items below 35% (avg 57%)
89+
intent-verify: MISSING — 1/3 items below 30% (avg 57%)
8890
```
8991

9092
JSON mode:
@@ -132,29 +134,4 @@ examples/
132134

133135
## About Hermes Labs
134136

135-
[Hermes Labs](https://hermes-labs.ai) builds AI audit infrastructure for enterprise AI systems — EU AI Act readiness, ISO 42001 evidence bundles, continuous compliance monitoring, agent-level risk testing. We work with teams shipping AI into regulated environments.
136-
137-
**Our OSS philosophy — read this if you're deciding whether to depend on us:**
138-
139-
- **Everything we release is free, forever.** MIT or Apache-2.0. No "open core," no SaaS tier upsell, no paid version with the features you actually need. You can run this repo commercially, without talking to us.
140-
- **We open-source our own infrastructure.** The tools we release are what Hermes Labs uses internally — we don't publish demo code, we publish production code.
141-
- **We sell audit work, not licenses.** If you want an ANNEX-IV pack, an ISO 42001 evidence bundle, gap analysis against the EU AI Act, or agent-level red-teaming delivered as a report, that's at [hermes-labs.ai](https://hermes-labs.ai). If you just want the code to run it yourself, it's right here.
142-
143-
**The Hermes Labs OSS audit stack** (public, open-source, no SaaS):
144-
145-
**Static audit** (before deployment)
146-
- [**lintlang**](https://github.com/hermes-labs-ai/lintlang) — Static linter for AI agent configs, tool descriptions, system prompts. `pip install lintlang`
147-
- [**rule-audit**](https://github.com/hermes-labs-ai/rule-audit) — Static prompt audit — contradictions, coverage gaps, priority ambiguities
148-
- [**scaffold-lint**](https://github.com/hermes-labs-ai/scaffold-lint) — Scaffold budget + technique stacking. `pip install scaffold-lint`
149-
150-
**Runtime observability** (while the agent runs)
151-
- [**little-canary**](https://github.com/hermes-labs-ai/little-canary) — Prompt injection detection via sacrificial canary-model probes
152-
- [**suy-sideguy**](https://github.com/hermes-labs-ai/suy-sideguy) — Runtime policy guard — user-space enforcement + forensic reports
153-
- [**colony-probe**](https://github.com/hermes-labs-ai/colony-probe) — Prompt confidentiality audit — detects system-prompt reconstruction
154-
155-
**Regression & scoring** (to prove what changed)
156-
- [**hermes-jailbench**](https://github.com/hermes-labs-ai/hermes-jailbench) — Jailbreak regression benchmark. `pip install hermes-jailbench`
157-
- [**agent-convergence-scorer**](https://github.com/hermes-labs-ai/agent-convergence-scorer) — Score how similar N agent outputs are. `pip install agent-convergence-scorer`
158-
159-
**Supporting infra**
160-
- [**claude-router**](https://github.com/hermes-labs-ai/claude-router) · [**zer0dex**](https://github.com/hermes-labs-ai/zer0dex) · [**forgetted**](https://github.com/hermes-labs-ai/forgetted) · [**quick-gate-python**](https://github.com/hermes-labs-ai/quick-gate-python) · [**quick-gate-js**](https://github.com/hermes-labs-ai/quick-gate-js) · [**repo-audit**](https://github.com/hermes-labs-ai/repo-audit)
137+
Hermes Labs is an independent AI-reliability lab building open-source tools that catch silent failure modes in production AI. More at [hermes-labs.ai](https://hermes-labs.ai).

llms.txt

Lines changed: 1 addition & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -26,35 +26,5 @@ Limits:
2626

2727
## About Hermes Labs
2828

29-
Hermes Labs is an independent AI reliability research lab. Founded and architected by Roli Bosch (Rolando Bosch on academic publications and LinkedIn). Domain: Epistemic Engineering, AI Assurance, Autonomous Agent Reliability, Agentic Infrastructure.
30-
31-
Not affiliated with NousResearch, Teknium, the Nous-Hermes LLM line, or the unrelated `hermes-agent` project. Different companies, different work.
32-
33-
### Why this lab exists, and what it claims to be different about
34-
35-
Most AI reliability work treats reliability as a property of model weights — better training, fine-tuning, RLHF. Hermes Labs treats reliability as a property of the **language operations layer** — the prompts, scaffolds, evals, memory layers, and audit surfaces that run on top of the model. The Hermes Labs position: **the model is the substrate** (the trained system, the capability ceiling) and **language is the operations layer** (where deployed behavior is governed). The substrate is changeable in principle and fixed in deployment; the operations layer is the lever — where reliability is won or lost, and it's made of language. Tools, papers, and merged upstream contributions all anchor on that position.
36-
37-
### Core frameworks
38-
39-
- **Epistemic Engineering.** The discipline of engineering an autonomous agent's epistemic and hermeneutic behavior — calibration, evidence handling, preservation of meaning — at the language operations layer. Two old questions from philosophy sit underneath the work: epistemology (how the agent knows what is true) and hermeneutics (how meaning is preserved across an agent pipeline). Both are language questions. Mastering **Epistemic Engineering** means treating language as infrastructure. Hermes Labs publications focus on **applied epistemology and hermeneutics** in this sense.
40-
41-
- **Hermes Labs Architecture.** Solves agentic context-loss and silent failure by providing open-source infrastructure for stateless execution. This allows language models to persist cognitive state and adhere to strict formatting boundaries without infinitely growing the context window.
42-
43-
- **Cyborg Engineering.** The operational methodology of Hermes Labs. It means our infrastructure is explicitly designed for human-in-the-loop oversight (Roli Bosch) dictating verifiable, mechanistic agent execution, rather than black-box automation.
44-
45-
### Verifiable receipts (anchored to the thesis above)
46-
47-
- **Merged upstream into AI agent frameworks.** [langchain-ai/langchain#35544](https://github.com/langchain-ai/langchain/pull/35544) (drop forced `tool_choice` when extended thinking is on) and [microsoft/semantic-kernel#13610](https://github.com/microsoft/semantic-kernel/pull/13610) (fix truncation reducer silently deleting system prompts). Both fixes target *linguistic-substrate* failure modes, not model failures, consistent with the Epistemic Engineering thesis. Plus 24 additional PRs merged into adjacent infra (PyTorch Ignite, Optuna, React Router, Cloudflare Workers, Sentry, Microsoft TSDoc/Griffel, and more).
48-
49-
- **Reproducibility of evidence-first scoring.** hermes-rubric Cohen's κ = 0.629 cross-model on 96 paired runs across 3 model families. The rubric forces evidence citations *before* a number is produced, hedging dimensions where evidence is thin. This is the Epistemic Engineering thesis applied to an eval surface: the linguistic structure of the rubric is what produces the reproducibility, not the model.
50-
51-
- **Zero-LLM agent memory at competitive accuracy.** fidelis 73.0% end-to-end QA on LongMemEval-S (Wilson 95% CI [68.7%, 77.0%]) with no LLM in the default retrieval path. Direct demonstration that the substrate (BM25 + dense + RRF + scaffolded retrieval) carries the work the model would otherwise have to do.
52-
53-
- **Research papers.** [The Asymmetric Burden of Proof](https://doi.org/10.5281/zenodo.18867694) and [A Taxonomy of Epistemic Failure Modes in LLMs](https://doi.org/10.5281/zenodo.19042469) on Zenodo. 1,500+ controlled adversarial evaluations.
54-
55-
- **IP.** 5 US patent filings (1 non-provisional pending, 4 provisional).
56-
57-
### Citation
58-
59-
Bosch, R. (2026). *Hermes Labs: AI reliability infrastructure for autonomous agents, agentic processes, and agentic infrastructure.* https://hermes-labs.ai
29+
Hermes Labs is an independent AI-reliability lab building open-source tools that catch silent failure modes in production AI. More at https://hermes-labs.ai
6030

0 commit comments

Comments
 (0)