Skip to content

feat: add Nobulex conformance implementation (Python, Ed25519) - #12

Open
arian-gogani wants to merge 2 commits into
ScopeBlind:mainfrom
arian-gogani:add-nobulex-implementation
Open

feat: add Nobulex conformance implementation (Python, Ed25519)#12
arian-gogani wants to merge 2 commits into
ScopeBlind:mainfrom
arian-gogani:add-nobulex-implementation

Conversation

@arian-gogani

@arian-gogani arian-gogani commented Jul 12, 2026

Copy link
Copy Markdown

Adds Nobulex as an implementation of draft-farley-acta-signed-receipts, per the invitation in
arian-gogani/nobulex#21 by @tomjwxf.

Where this stands, stated plainly: this PR does not pass the suite, and I would rather say so
than let it keep sitting. The driver here emits the v1 flat shape, which passes check 1 on all four
fixtures and comes back unknown_format from check 2.

Chasing that turned into #13. The short version is that the two checks accept disjoint sets: both
shapes expected/receipt-schema.json defines are unknown to @veritasacta/verify, and the
decision_receipt shape the verifier does recognize is rejected by the schema, including on this
repository's own reference receipts. The measurement is in the issue and it reproduces from a clean
clone.

So the shape this PR should emit is not mine to pick. I have a driver ready that emits
decision_receipt and comes back clean from the verifier, and I will force-push it here the moment
the schema question in #13 has an answer. If you would rather I push it now and let the schema catch
up afterwards, say so and it goes up today.

What the driver produces: four receipts, one per fixture, Ed25519 over JCS-canonical bytes
(RFC 8785), chain-linked via parent_receipt_hash.

Seq Tool Decision Signature Chain
1 Read allow verified genesis
2 Bash allow verified linked
3 Bash deny verified linked
4 Write allow verified linked

Where this canonicalizer has been checked by other people, since that is the part that matters
for a conformance registry and it is better coming from them than from me:

  • aeoess/agent-passport-system records 14 of 14 byte-match checks passing on 2026-05-02, ten APS
    JCS vectors plus four CTEF vectors, run through this canonicalizer against their own pinned
    canonical_sha256 values. Their write-up is tests/fixtures/BYTE-MATCH-VERIFICATION.md.
  • AgentAvow/AgentAvow v0.3.2 lists @nobulex/crypto among the byte-match validated
    implementations of the CTEF v0.3.1 envelope.
  • FransDevelopment/open-agent-trust-registry carries an issuer entry with the Ed25519 key.
  • aeoess/agent-governance-vocabulary carries a crosswalk mapping the primitives to their canonical
    vocabulary.

The action_ref construction is merged normative implementation guidance in the OWASP Agentic
Skills Top 10 under AST09.

Correction to the original description of this PR: it claimed a citation in an x402 conformance
spec. That was wrong. The 14 of 14 figure is from aeoess/agent-passport-system as described above
and has nothing to do with x402. I have removed the claim rather than leave it standing.

4/4 vectors pass: signatures verified, chain integrity verified,
decisions match expected (allow/allow/deny/allow).

Nobulex is credit scores for AI agents. The action_ref formula is
normative implementation guidance in OWASP Agentic Skills Top 10
(AST09, PRs #35/#38). Cited as third independent issuer in x402.

PyPI: pip install nobulex
npm: @nobulex/core
GitHub: github.com/arian-gogani/nobulex
@arian-gogani arian-gogani changed the title feat: add Nobulex conformance implementation (Python, Ed25519, v1 flat) feat: add Nobulex conformance implementation (Python, Ed25519) Aug 26, 2026
@arian-gogani

Copy link
Copy Markdown
Author

Updating this rather than letting it keep sitting, since the reason it has not moved turned out not
to be the PR.

I went back through the suite from a clean clone and opened #13. The part that matters here is that
the two checks accept disjoint sets of receipts. Both shapes expected/receipt-schema.json defines
come back unknown_format from @veritasacta/verify, and the decision_receipt shape the verifier
does recognize fails the schema, including on the reference receipts in aps-gateway-enforcement/.
Measured, reproducible from a clean clone, written up in the issue.

This PR emits v1 flat, so it is on the side that fails check 2. A driver emitting decision_receipt
is ready and comes back clean from the verifier. I have not pushed it because it would be an
implementation deciding what the suite accepts, which is your call. Say the word either way and I
will push it or hold.

Separately, ./conformance/run.sh returns 0 even when it prints NON-CONFORMANT, so CI has been green
this whole time on a suite where nothing passes. That one is a single line and is the first item in
the issue.

@tomjwxf

tomjwxf commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

This is a considerable favour and I want to be clear about that before anything else. You went looking, found six real things, measured all of them, and held a working driver rather than push a change that would have quietly decided what the suite accepts. That last part is the bit most people would not have done.

I reproduced everything I checked from a clean clone. You were right on all of it.

The decision you are blocked on

Build against decision_receipt. Push the driver.

The schema is what is wrong here, not your driver and not the reference receipts. expected/receipt-schema.json requires pubkey and payload.type matching ^scopeblind\.receipt\.v\d+$, while the receipts this repository actually ships in aps-gateway-enforcement/ carry kid and a top-level type: decision_receipt. So the schema rejects our own reference receipts, and @veritasacta/verify accepts a shape the schema has never described. Of the two, the receipts and the verifier are the ones with users.

If you want to send the schema and check 1 change alongside the driver, please do. You are not deciding it unilaterally at that point, I am asking for it. If you would rather keep the PR to the driver, say so and I will do the schema separately and you can rebase.

You are also right that the verifier reports -03 while the fixtures cite -02. That is a real inconsistency on my side and I will reconcile it rather than have you code against a moving target.

Fixed in #14

Findings 1, 3 and 5, with your reproductions:

run.sh now returns its verdict. On the current tree it exits 1 and prints one implementation verified, four failed, which is the true state and should have been visible long ago.

Check 2 passes --key from the published seed, so no_public_key stops being reported as a tampered signature.

Check 3 now reads expected/chain.jsonl, compares tool_name, decision and policy_id, and compares the parent hash against the predecessor digest instead of computing it and discarding it. It also cross-checks chain.jsonl against the fixtures' expected_decision so those two cannot drift apart unnoticed.

Demonstrated rather than asserted: on a receipt set that allows the destructive Bash fixture the policy forbids, the old check 3 exits 0 and the new one exits 1.

Finding 2 is separate and also correct. There is no sign subcommand in the protect-mcp CLI at all, so the reference driver could never have produced anything. Fixing that next.

On your last paragraph

You wrote that a suite whose checks pass on receipts nobody evaluated is the same failure the receipts are meant to prevent, one level up. That is the correct criticism and it is the one I should have caught myself. A conformance suite that reports green while nothing conforms is worse than no suite, because it converts an open question into a false answer.

Thanks for going after it properly rather than working around it.

…hape in check 1

Per the maintainer's request in ScopeBlind#12: build against decision_receipt, and send
the schema and check 1 change alongside the driver.

Driver
------
Evaluates fixtures/policy/autoresearch-safe.cedar and never reads
expected_decision. cedar_lite.py is a small evaluator for the subset of Cedar
that policy uses, not a Cedar implementation, and says so at the top of the
file. A driver that reads the fixture's expected answer copies the answer key,
which is finding 6 in ScopeBlind#13.

The parent-hash canonical form is byte-identical to the one conformance check 3
computes, so a mismatch means disagreement about decisions rather than
disagreement about bytes.

No inline public key. draft-farley-acta-signed-receipts says a verifier must
not accept a key carried inside the envelope unless it is independently
anchored, so kid names the key and resolution is the verifier's. kid is the
RFC 7638 thumbprint of the shared conformance seed's public half, which is why
it matches the kid on the aps-gateway-enforcement receipts.

Output is deterministic: the four receipts hash byte-identically when generated
on Linux and on macOS.

Schema and check 1
------------------
expected/receipt-schema.json gains a decisionReceipt branch, and its description
is corrected. It previously claimed @veritasacta/verify accepts both the v1 flat
and v2 envelope shapes; it accepts neither and returns unknown_format for both.
That sentence is where finding 4 in ScopeBlind#13 comes from, and the consequence was that
the schema rejected this repository's own aps-gateway-enforcement receipts. They
validate now.

check 1 in conformance/verify.sh recognises the same shape, kept dependency-free
in the existing style.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GSeQ2tJWZkD5NwQkr6J5iC
@arian-gogani

Copy link
Copy Markdown
Author

Pushed. Built against decision_receipt as you asked, and I took you up on sending the schema and check 1 change alongside it.

What is in the push

The driver. emit.py parses fixtures/policy/autoresearch-safe.cedar and evaluates each fixture against it. It never reads expected_decision; the fixture's copy is popped before use so a later edit cannot quietly start depending on it. cedar_lite.py is a small evaluator for the subset of Cedar that policy uses, not a Cedar implementation, and says so at the top. That was the only way to satisfy finding 6 honestly: a driver that reads the answer key would now pass check 3 while proving nothing.

Its parent-hash canonical form is byte-identical to the one check_chain.py computes, deliberately. If it differed, every parent hash would mismatch and the failure would look like tampering rather than like disagreement about bytes.

No inline public key. kid names the key. It comes out as 3iR-H6Xx_3rpt7eNMUVNazSZkUclb_cekBJZZL4mlUs, the same kid your aps-gateway-enforcement receipts carry, because both are the RFC 7638 thumbprint over the same seed. That was a useful accident: it confirmed independently that we derive it the same way.

Schema and check 1. expected/receipt-schema.json gains a decisionReceipt branch. I also corrected its description, which said @veritasacta/verify accepts both existing shapes. It accepts neither and returns unknown_format for both, and that sentence is where finding 4 came from. Your three aps-gateway-enforcement reference receipts validate against the schema now; they did not before.

What it does on each tree

tree result
main as it stands 5 passed, 1 failed. Check 2 has no --key, so it reports no_public_key. That is finding 3, which #14 fixes.
main + #14 all three checks pass

So this PR is green once #14 lands, and I would rather it merged in that order than have you work around check 2 for me.

The suite still catches a bad driver

Adding a shape to check 1 could easily have made it permissive, so I planted seven failures and measured the exit code directly rather than through a pipe:

  clean              -> exit 0  (6 passed, 0 failed)
  tampered sig #1    -> exit 1
  tampered sig #2    -> exit 1
  tampered sig #3    -> exit 1
  tampered sig #4    -> exit 1
  wrong decision     -> exit 1  (check 3)
  broken chain link  -> exit 1  (checks 2 and 3)
  junk shape         -> exit 1  (checks 1 and 3)

Output is deterministic. The four receipts hash byte-identically generated on Linux and on macOS.

Finding 7, and it is in check 2

Getting that matrix is how I found it. On the first run, three of the four tampered-signature cases passed.

Check 2 hands the verifier a glob:

npx --yes @veritasacta/verify --key "$CONFORMANCE_KEY" "$RECEIPTS_DIR"/*.json

Its usage line takes a single <file.json>. Given four positionally, it verifies only the last one. Measured by tampering each position in turn against the four-receipt directory:

  tampered receipt-0001 -> exit 0
  tampered receipt-0002 -> exit 0
  tampered receipt-0003 -> exit 0
  tampered receipt-0004 -> exit 1

It prints exactly one Signature: line for four input files. Alone, the tampered file exits 1 correctly, so the verifier is fine; the invocation is not.

The consequence is the same shape as the others. An implementation can forge three of its four receipts and check 2 reports "all signatures verify." This is on main and it is still there in #14, since #14 adds --key to the same glob.

The fix is a loop, one invocation per file, with the pass line moved after it. I have it working locally and left it out of this PR because check 2 is yours in #14 and I did not want to conflict with you. Say the word and I will send it as a separate PR, or take it into #14 yourself, whichever you prefer.

I have added it to #13 as finding 7 with the position-by-position numbers.

On the -02 versus -03 reconciliation

No rush from my side. The driver signs over the whole envelope minus signature, hex encoded, which is what your reference receipts do and what the verifier accepts today. If reconciling moves that, tell me and I will follow it rather than pin to whatever happens to work now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants