Skip to content

Commit f3edfa3

Browse files
authored
docs(specs): 101-tpa-db — versioned signed offline TPA signature database (#1028)
* docs(specs): add 101-tpa-db — versioned signed offline TPA signature database Spec-only (speckit stage 1) for the roadmap P1 tpa-db epic: signed, sequence-versioned bundle format with anti-downgrade + rollback; loader verify-before-parse extending the 086 fail-closed pipeline; seed corpus strategy (>=25 signatures, provenance/license, eval sample pairs); the publication channel riding the spec-087 refresh lifecycle; post-activation informational re-scan; freshness telemetry. Eval-gated on the existing scan-eval recall/FP CI bar. * docs(specs): close trust-model and eval-gate gaps in 101-tpa-db Cross-model review (opencode/gpt-5.6-sol) plus local verification against the shipped loader and eval harness found ten genuine gaps in the stage-1 spec. All were confirmed against the code before being addressed. ## Changes Trust model / anti-downgrade - FR-009: the sequence floor is now a property of SIGNED artifacts only. `sequence` lives in the manifest, so an unsigned drop's sequence is attacker-chosen; an unsigned candidate could previously claim a huge sequence and permanently lock out every legitimate signed release. Unsigned candidates no longer advance the floor, and a signature ratchet stops signing from being stripped off an install that already ran signed. - FR-010: rollback no longer lowers the floor. Resetting the floor to the rolled-back sequence meant the next refresh could immediately re-activate the exact release the operator just rejected. Rollback is now a pinned exception over a monotonic high-watermark, with the rejected (sequence, fingerprint) deny-listed. - FR-009a: the embedded-default availability fallback is stated as the single explicit exemption to SC-002 (it previously contradicted it outright) and must serve as DEGRADED coverage, so deleting the external bundle cannot be used to obtain scan-mode auto-approvals under a weaker corpus. - FR-007: an active unsigned/unverified bundle likewise marks coverage degraded. `inprocess.go` derives `coverageOK` from bundle *presence*, so without this an attacker-supplied unsigned corpus keeps auto-approving. - FR-003: key ids must be canonical public-key fingerprints, duplicates rejected, and manifest id / sidecar id / verifying-key id must all match. - FR-006/FR-008: corrected an impossible pipeline order — the spec required checking the manifest-resident `sequence` before parsing the manifest. Verification is what precedes parsing; it runs over the raw bytes that are then parsed from memory, closing a verify/parse swap race. - FR-011a: activation must be crash-consistent. Spec 087 FR-010's atomicity is in-memory only; a crash between activation and floor persistence reopens the replay window FR-009 exists to close. Eval gate (the gate was measuring nothing) - FR-016a/SC-005/US2: `cmd/scan-eval/gate.go`'s `gateChecks()` registers only the four built-in detect checks — `scanner.BundleCheck` is absent and scan-eval never loads a bundle, so FR-016's "merge-blocking on corpus change" and SC-005 would both have passed vacuously. The harness must load a candidate through the production loader, register the bundle check, and map each new campaign category (`gatedCategory()` enforces a category only when its mapped check id is registered). SC-005 gains a negative control. Telemetry - FR-021: the schema-v8 anonymity backstop hard-whitelists `tpa_scanner` to four integer counters, so string/bool bundle fields would be rejected before transmit. Requires typed whitelist entries, an enum-bounded `source`, and publisher-issued `bundle_version` values only (a self-built corpus version string would itself be a fleet identifier). Consistency - FR-007: states explicitly that this spec supersedes Spec 087 FR-014's precondition, since embedding publisher keys means a fetch is no longer refused merely because the operator configured no key of their own. Related #1028 * docs(specs): fix watermark lockout + embedded-default regression in 101-tpa-db Round-2 cross-model review (opencode/gpt-5.6-sol) on the round-1 revisions. Six findings, all verified genuine; two of them were regressions the round-1 fixes introduced. ## Changes Watermark could be weaponized into a permanent update lockout - FR-005a (new): the trust set carries a monotonic trust EPOCH that only a binary release can advance, and anti-downgrade is scoped (authority, epoch, sequence). Without it, key-compromise recovery was impossible rather than merely painful: an attacker holding a compromised publisher key signs at the maximum representable sequence, and since round-1 made the watermark monotonic, no surviving key could ever outbid it — a one-shot permanent denial of updates. FR-005 and the key-rotation edge case now route compromise recovery through the epoch bump. - FR-004a (new): anti-downgrade state is tracked per SIGNING AUTHORITY, not in one global namespace. An operator with a self-built signed corpus (an explicitly supported persona) would otherwise raise the single global watermark past the publisher's release line and permanently block official updates. All publisher keys group into one authority so per-authority watermarks stay compatible with key rotation. - SC-002a (new): both lockout scenarios must be proven by fixture. Regressions introduced by the round-1 fixes - FR-007/FR-009a: round-1 made "any unsigned active bundle" degrade coverage, but the embedded default has no sidecar — so every fresh install would have shipped with scan-mode auto-approval silently disabled. Degradation now applies to unsigned EXTERNAL candidates only; the embedded default is binary-trusted and degrades solely when serving BELOW a retained signed watermark. - FR-008: round-1 put the rule-count ceiling before signature verification and parsing, which is impossible — rule count is not knowable from raw bytes. The two ceilings are now split: byte size before verification, rule count after parse. Consistency - FR-011a: the crash-consistent transaction now explicitly includes the watermark fingerprint and the FR-010 deny-list. Omitting them let a crash preserve a rollback while losing the state that rejects the rolled-away release, silently undoing the rollback on the next refresh. - Key Entities: "Sequence Floor" still said the floor is "lowered only by explicit rollback", directly contradicting revised FR-010. Renamed to Sequence Watermark, defined as monotonic, with rollback as a pinned exception. Terminology normalized from floor to watermark throughout so the two do not read as separate mechanisms. - Added a threat-model boundary note: an attacker with config write is out of scope (they can already disable quarantine), which is why the ratchet keys on install state and require_signed_bundle is config-resident. Related #1028 * docs(specs): bind the trust epoch into signed bundle bytes in 101-tpa-db Round-3 cross-model review (opencode/gpt-5.6-sol) found one CRITICAL defect, verified genuine: the trust epoch introduced in round 2 was itself a replay window. ## The defect FR-005a made the epoch a property of the binary's trust set only; FR-002's manifest carried sequence and key id but no epoch. After a compromise recovery bumped the epoch and re-baselined the watermark, an OLD, weaker bundle legitimately signed by a still-trusted surviving key would be reinterpreted as belonging to the NEW epoch, clear the fresh (near-zero) watermark, and activate with signature_verified=true and full coverage — reinstating precisely the blind spots the recovery release was published to close. SC-002a tested recovery but not this replay. ## Changes - FR-002: the trust epoch is now carried inside the SIGNED manifest, so it is cryptographically bound to the bundle and cannot be reinterpreted. - FR-005a: epochs are per-authority, and a candidate is accepted only when its manifest epoch EQUALS the current binary-trusted epoch of the verifying authority — never merely less-than-or-equal. A below-epoch bundle is refused as stale-epoch replay even when its signature verifies against a still trusted key. Documents the accepted consequence: a binary on the old epoch stops receiving database updates until updated, which is correct because it still trusts the compromised key, and must surface as a stale-epoch degraded state rather than failing silently. - FR-008: epoch match added to the pipeline, before the anti-downgrade stage. - SC-002a: new fixture (a2) — a pre-bump bundle signed by the surviving key is refused after the bump rather than clearing the re-baselined watermark. - FR-020: stale-epoch replay added to the surfaced rejection reasons; bundle status also reports signing authority and trust epoch. Also, from a self-audit of the round-2 per-authority model: - FR-009a: the embedded default degrades when below a watermark retained by ANY authority (previously ambiguous which authority was meant). - FR-004a: choosing an operator-signed corpus over a higher-sequence publisher one is a deliberate operator decision, not a downgrade and not degraded coverage — but the active authority must be visible in bundle status. Related #1028 * docs(specs): seed the anti-downgrade watermark from a trusted baseline (101-tpa-db) Round-4 cross-model review (opencode/gpt-5.6-sol) found one HIGH defect, verified genuine: the anti-downgrade guarantee did not exist at first use. ## The defect Every anti-downgrade mechanism in the spec binds only once persisted state exists. On a fresh install — or after any data-directory reset — there is no watermark, so an attacker who can write the bundle path drops an OLD but genuinely publisher-signed, current-epoch bundle (sequence 5 while the shipping release is at 100). Signature verifies, epoch matches, nothing to compare the sequence against: it activates as a fully signature-verified bundle at FULL coverage, silently reinstating every blind spot fixed between 5 and 100. The FR-009 ratchet and the FR-010 pin/deny-list do not help, since both also require pre-existing state. Confirmed no new contradiction was introduced by the round-3 epoch fix. ## Changes - FR-009b (new): watermarks are SEEDED from a trusted baseline rather than starting empty. The publisher authority seeds from the sequence of the bundle embedded in the running binary — a candidate older than what the binary already ships is strictly worse than the fallback and is refused — and the baseline is re-applied on every startup, so wiping the data directory cannot lower it below the binary's own corpus. The operator authority, which has no binary-trusted baseline, gets a minimum sequence configurable alongside the operator key. This also makes the FR-005a epoch re-baseline safe: a bumped epoch starts at the embedded sequence, not zero. - SC-002 / US1 Independent Test: fresh-install and post-wipe downgrade are now explicitly in scope, with fixture (h) covering the old-but-validly-signed bundle on an install with no persisted state. - Edge case "Anti-downgrade state lost": corrected — a wipe resets the watermark to the seeded baseline, not to zero. What a wipe genuinely loses is the ratchet state and any pin/deny-list. - Key Entities: Sequence Watermark is documented as never empty. Related #1028 * docs(specs): make the operator-authority baseline explicit and durable (101-tpa-db) Round-5 cross-model review (opencode/gpt-5.6-sol) found one HIGH defect, verified genuine. It confirmed FR-009b closes the publisher-side replay without contradicting FR-005a, FR-009, FR-009a, or FR-010, but caught that the operator authority was left with the same hole. ## The defect FR-009b gave the operator authority a minimum sequence "defaulting to zero, with the residual exposure being the already-out-of-scope config-write attacker". That parenthetical was wrong. With a zero default, no config write is needed: after a data-directory wipe (or on a fresh install), a bundle-path attacker — squarely inside the threat model — can replay any older genuinely operator-signed bundle at full coverage. The operator authority therefore had no post-wipe anti-downgrade at all by default, while the spec read as though it did. ## Changes - FR-009b: the operator baseline must be declared explicitly in mcp_config.json alongside the operator key, not defaulted to zero and not stored in the data directory — a baseline held in resettable state gives no protection exactly when it is needed. If an operator key is configured without a baseline, operator-authority anti-downgrade is treated as unsupported and coverage is marked degraded while an operator-signed bundle is active, instead of presenting it as fully trusted. The data-directory watermark may only raise the effective minimum above the configured baseline, never lower it. Epoch re-baseline now starts at the authority's baseline rather than zero. - Edge case "Anti-downgrade state lost": states plainly that the operator-side guarantee survives a wipe only with a config-resident baseline. - SC-002a: fixture (c) covers the operator authority after a wipe, both with and without a declared baseline. Note: per the repo's per-PR cross-model review cap, this is the fifth round. This fix has NOT itself been re-reviewed. Related #1028 * docs(specs): close the authority-swap hole and de-vacuate the gate control (101-tpa-db) Cross-model review (opencode / gpt-5.6-sol) round 1 on specs/101-tpa-db/spec.md. All five findings verified against the current code before changing anything. ## Changes - FR-004b (new): the active signing authority is a config-resident declaration, default publisher; a candidate signed by a non-active authority is refused ("authority not active"). Per-authority watermarks alone let a bundle-path attacker — in the threat model — swap a publisher bundle at sequence 100 for any operator-signed artifact clearing the operator baseline, silently narrowing coverage while still reporting signature_verified=true at full coverage, because the comparison never crosses namespaces. - FR-009a: the embedded default's sequence is compared ONLY against the publisher watermark (same namespace); the "any authority" rule made an operator corpus at sequence 900 permanently degrade a current publisher fallback at 100. The intent behind it is restated directly as condition (b): a previously-active external bundle of the active authority having gone missing. - FR-016a: corrected the claim about unmapped eval categories — they are excluded from OverallRecall so their MISSES cannot fail the gate, but their false positives still count, since FPRate is computed over the whole hard-negative set regardless of category mapping (gate.go:193-197, 287-289). - FR-016b (new) + SC-005 + US2 independent test: the anti-vacuity negative control is made deterministic. Aggregate recall over 25+ signatures absorbs a single removal within the 0.10 threshold headroom, and built-in checks (phrase.injection especially) often flag the dropped signature's sample anyway, so "remove a gating signature and the gate must go red" could pass vacuously. A designated control signature with no built-in or sibling overlap, an asserted gate-breach exit code, and a CI assertion of the no-overlap property replace it. - SC-001: tampering is defined over the fields the detached signature actually covers (bundle bytes, signature value, sidecar identity fields, mismatched pair) instead of "any byte of ... sidecar", which no detached scheme can guarantee against whitespace/key-order edits. - SC-002a gains fixture (b2) and US1's independent test gains case (i) for the authority-not-active refusal; FR-020, the threat-model edge case, Key Entities, and the Constitution Check note the new config-resident field. ## Testing - Docs-only change; no Go sources touched. - Every codebase claim re-verified before editing: cmd/scan-eval/gate.go (gateChecks/categoryCheck/OverallRecall/FPRate, no --bundle flag today), internal/security/scanner/inprocess.go (coverageOK), and internal/telemetry/anonymity.go (schema-v8 tpa_scanner whitelist). * docs(specs): bind rollback to a fingerprint and narrow the degraded fallback (101-tpa-db) Cross-model review (opencode / gpt-5.6-sol) round 2 on specs/101-tpa-db/spec.md. All three findings verified against the document and gate.go before changing anything. ## Changes - FR-010: the rollback target must be identified by content fingerprint, not by pathname alone. Rollback is the one path that deliberately bypasses the watermark, so naming its target by an attacker-writable path lets a bundle-path attacker choose what the bypass lands on — substituting a genuinely publisher-signed, current-epoch bundle at sequence 5 for the sequence-90 artifact the operator intended, which then gets PINNED. The operator now supplies the expected fingerprint, the path is only a hint, a mismatch is refused, and a rollback target must sit at or above the FR-009b seeded baseline (the embedded default excepted, since it is that baseline). - FR-009a condition (b): narrowed to a missing OPERATOR-authority bundle. As written it degraded coverage whenever any previously-activated external bundle went missing, including the ordinary case of a publisher external file removed after a binary upgrade shipped a strictly newer embedded corpus — where condition (a) already answers the question on the same axis and says full coverage. - US2 independent test: the negative control now runs over the FR-016b control scope instead of re-running the full-corpus command. One miss out of a 25-signature corpus leaves aggregate recall near 0.96, above the 0.90 threshold, so the full-corpus form contradicted FR-016b and would have proven nothing. - US1 independent test, SC-002, FR-020: assertions and rejection reason for the rollback-target mismatch. ## Testing - Docs-only change; no Go sources touched. - Recall/FP semantics re-checked in cmd/scan-eval/gate.go (OverallRecall is gatedDetected/gatedMalicious aggregated across gated categories). * docs(specs): cap the sidecar, scope the rollback floor, pin the gate control (101-tpa-db) Cross-model review (opencode / gpt-5.6-sol) round 3 on specs/101-tpa-db/spec.md. All three findings verified against the document and gate.go before changing anything. ## Changes - FR-011 + FR-008 + edge cases: the sidecar gets its own small byte cap and a bounded read, applied before it is parsed, on both the file-drop and fetch paths. The sidecar is the one artifact that must be read before any signature is verified — its algorithm id, key id, and signature value are verification's own inputs — so FR-006's verify-before-parse protection does not cover it, and a bundle-path attacker could drop a multi-gigabyte sidecar next to an otherwise valid bundle and stall or exhaust the refresh path. The pipeline order and the "two ceilings" note are updated accordingly. - FR-010: the rollback floor is now its OWN authority's FR-009b baseline, not the publisher one in both cases. As written it compared an operator-signed rollback target at sequence 9 against the publisher embedded corpus at 100 and would have refused every legitimate operator rollback, contradicting the per-authority isolation of FR-004a. - FR-016b: "restrict the control to its category" is not by itself provably failing — decide() breaches only on OverallRecall < minRecall, so a control category with ten gated malicious samples lands on exactly 0.90 after one miss and stays green on the boundary. The canonical construction is now a control corpus whose gated malicious set is the control sample alone; any other scope must compute and assert the recall drop rather than assume it. ## Testing - Docs-only change; no Go sources touched. - Boundary semantics re-checked in cmd/scan-eval/gate.go decide() (strict `<` against min-recall) and the per-authority baseline rules in FR-009b. * docs(specs): refuse non-regular candidates and scope the freshness metric (101-tpa-db) Cross-model review (opencode / gpt-5.6-sol) round 4 on specs/101-tpa-db/spec.md. Findings verified against the document before changing anything; the telemetry finding was accepted in substance but not in its proposed remedy (see below). ## Changes - FR-011 + Edge Cases: a byte cap bounds bytes, not time. Local candidate artifacts (bundle and sidecar) must be REGULAR files — a FIFO, device, socket, or symlink resolving to one is refused outright rather than read — and every candidate read, local or fetched, runs under an enforceable I/O deadline whose expiry releases Spec 087's single-flight slot. Otherwise the bundle-path attacker leaves a FIFO with no writer and pins the refresh path indefinitely without ever approaching the cap. - FR-010: an authority with no declared baseline — possible only for the operator authority, which FR-009b explicitly permits — has no rollback floor to enforce, and none is invented. The rollback proceeds and the install is marked degraded while that bundle is active, which is already FR-009b's treatment. Refusing the rollback would break FR-010 for a configuration FR-009b allows; assuming a zero floor would claim a guarantee that does not exist. - SC-006: the freshness metric is scoped to publisher-issued corpora and its derivation is stated — age comes from a server-side join of the reported version/sequence to the publisher's own release dates. Installs on an operator-built corpus (the `other` bucket) are not ageable and form their own bucket rather than being folded into the ratio. ## Notes on the rejected remedy The reviewer's suggested fix for SC-006 was to report `generated_at`/age in the heartbeat. That is exactly the near-unique value FR-021's schema-v8 backstop exists to keep out of the payload, and it would buy nothing the version/sequence join does not already give, so the derivation was specified instead. ## Testing - Docs-only change; no Go sources touched. * docs(specs): name the baseline-less exemption and bucket operator sequences (101-tpa-db) Cross-model review (opencode / gpt-5.6-sol) round 5 on specs/101-tpa-db/spec.md. Round 5 reported no remaining adversary-reachable security hole; both findings were internal contradictions, verified against the document before editing. ## Changes - FR-009b, the Sequence Watermark entity, and SC-002: the baseline-less operator authority is now named as an explicit exemption everywhere the document made a universal claim. FR-009b permits an operator authority with no declared baseline (marking the install degraded), while the same requirement opened with "every watermark MUST be SEEDED", the entity said "Never empty", and SC-002 called FR-009a the "sole exemption" — an implementation could not satisfy all four at once. There are exactly two exemptions and both are degraded-coverage states. - FR-021: `sequence` is reported only when `bundle_version` resolves to a publisher-issued value; in the `other` bucket it is omitted. Bucketing the version string while transmitting a raw operator-chosen sequence defeats the purpose of the bucket, since an operator-built corpus can number itself with a unix timestamp or private build number and be just as fleet-identifying. SC-006 already scopes freshness to publisher corpora, so no analysis is lost. ## Testing - Docs-only change; no Go sources touched. - Round 5 is the per-PR cross-model review cap (CLAUDE.md escalation trigger 4); these two fixes are not themselves re-reviewed. * docs(specs): correct the SC-002a fixture count after adding (b2) (101-tpa-db) * docs(specs): align FR-007 and FR-009a with the two-exemption rule (101-tpa-db) * docs(specs): scope pins per epoch, gate telemetry on provenance, fix exemption count (101-tpa-db) Cross-model review (opencode / gpt-5.6-sol) round 6 on specs/101-tpa-db/spec.md. All five findings verified against the document and the cited code before editing. - Edge Cases (threat model): the boundary is now explicit about the data directory — the bundle-path attacker may also be able to delete/reset the persisted security state (equivalent to the wipe), against which only the binary/config-anchored guarantees survive (seeded publisher baseline, config-resident operator baseline, require_signed_bundle, active-authority declaration); ratchet and pin/deny-list are best-effort. Chosen-value corruption of the state store is out of scope (that attacker is already past every mcpproxy guarantee). - Edge Cases + FR-009a: two stale 'single exemption to SC-002' statements now name both exemptions, matching SC-002's 'exactly two'. - US1 acceptance 3 + SC-002: 'lower sequence' is qualified everywhere as a same-authority, same-epoch comparison; a deliberate FR-004b authority switch is by definition not a downgrade and needs no exemption. - FR-010 + FR-005a: pin/deny-list entries are keyed (authority, epoch, sequence, fingerprint); an authority switch suspends (not deletes) the non-active authority's entries; an epoch bump retires prior-epoch entries so stale rollback state cannot block the key-compromise recovery bundle. - FR-021: 'resolves to a publisher-issued value' is a provenance test (embedded default or publisher-authority-verified), never a claimed-version-string match — an operator corpus can copy a publisher version string verbatim; the backstop also rejects sequence-present-with-version-other cross-field. - Docs-only change; no Go sources touched. - Round 6 of the raised (10-round) cross-model review cap. * docs(specs): define authority-switch serving semantics + operator epoch recovery (101-tpa-db) Cross-model review (opencode / gpt-5.6-sol) round 7 on specs/101-tpa-db/spec.md. All three findings verified against the document before editing. ## Changes - FR-004b: the active-authority declaration governs the SERVING bundle, not only future candidates. On a declaration change, an active external bundle of the now-inactive authority is deactivated (loudly logged) and the install falls back to the embedded default per FR-009a until a bundle of the newly active authority activates. Otherwise keep-last-known-good makes the switch advisory: a bundle-path attacker keeping the new authority's candidates invalid could perpetuate the old corpus at full coverage. Declaring the operator authority active does not by itself degrade coverage. - FR-009 + SC-002: the highest-ever watermark and the signature ratchet are explicitly claims about retained data-directory state — after a reset only the FR-009b seeded/config-resident baselines survive and the ratchet re-arms on the next signed activation. Fixtures must test the post-reset guarantees the threat model actually supports, resolving the contradiction with the 'Anti-downgrade state lost' edge case. - FR-005a + Key Entities: epoch advancement is split per trust root — binary release for the publisher epoch, config edit for the operator epoch (declared alongside the operator keys/baseline). A binary-only rule made operator-key compromise an unrecoverable permanent lockout, since no publisher release can coordinate a private operator authority's epoch and a replacement key inherits the same-epoch watermark. A config bump re-baselines from the config-declared baseline and retires prior-epoch operator pins/deny-list. ## Testing - Docs-only change; no Go sources touched. - Round 7 of the raised (10-round) cross-model review cap. * docs(specs): close cross-model review gaps in 101-tpa-db Cross-model review (opencode/gpt-5.6) against the shipped 086/087 code found seven defects that survived verification against the repo: - Threat model called config.db "integrity-protected". It is BBolt: transactional and checksummed, not authenticated. Restated as a trust boundary and required SUBSTITUTION fixtures (prepared db with chosen watermark / cleared ratchet / forged pin), not just deletion/reset. - FR-010 authorized a watermark-bypassing rollback with the fingerprint "offered by bundle status history". That fingerprint is hex(sha256)[:12] — 48 bits — and no activation history exists (BundleInfo reports only the live artifact). Now: full SHA-256, compared byte-for-byte, plus a durable append-only activation record specified as new work. - FR-011 required candidates to "be REGULAR files" and to read under an "enforceable I/O deadline". A pathname stat is a substitution window, and Go's SetReadDeadline is unsupported on regular files (ErrNoDeadline) — the rule was unimplementable as written. Now: O_NOFOLLOW open + fstat on the descriptor, and a bounded budget enforced outside the read. - FR-001 ("the existing compiled bundle") and FR-012 ("each signature MUST carry provenance and a redistributable license") could not both hold: rules[] has no such fields. Added FR-001a — an additive signatures[] section keyed by TPA id, many-to-one to rules, with referential-integrity validation. - FR-001's "byte-identical rebuilds for a given corpus" contradicted FR-002 binding release metadata into the signed bytes. Determinism now scoped to (corpus, manifest metadata) with pinned inputs. - sequence had no numeric domain while SC-002a relies on a maximum. Now a canonical uint64 as a JSON decimal string, with explicit rejection rules. - FR-003 left the sidecar wire format undefined although it is the one artifact parsed before any signature is verified. Design must fix schema, encodings, required fields and rejection behaviour; Ed25519-only in v1. SC-010..SC-012 added so each new requirement is testable. * docs(specs): put numbers on the 101-tpa-db load ceilings FR-011 deferred every threshold to "design" — the bundle byte size, the rule count, the sidecar cap ("sized to its fixed small structure") and the I/O deadline. Each of those is a rejection boundary, and a rejection boundary with no number cannot be tested: no fixture can be built one byte or one rule over a value that does not exist yet, so the requirement would have shipped unfalsifiable. It is also not a tightening of something already there. Today the loader has no ceiling of any kind: tpa_bundle_source.go reads the candidate with a bare os.ReadFile(path) — no size check and no file-type check — and loadBundleCheck applies no rule-count limit. Numbers now in the FR: bundle <= 8 MiB, rules <= 2000 (SC-008 exercises >= 500, so four more corpus generations fit without a spec change), sidecar <= 4 KiB, read budget 5 s per candidate. At the ceiling loads, one over is rejected whole. SC-010a drives both sides of every boundary and asserts the last-known-good corpus keeps serving. * docs(specs): 101-tpa-db implementation plan Phase 0 + Phase 1 for the tpa-db epic: research.md (R1-R11 findings, D1-D11 decisions), plan.md, data-model.md, three contracts, quickstart.md. The research pass changed the shape of the plan in three ways worth calling out, each because the tree does not look like the documents describe it: - Spec 087's plan names a package internal/security/bundle that does not exist. What shipped (086) lives in internal/security/scanner/tpa_bundle_source.go and tpa_bundle.go. Reading 087's plan as a description of the tree is the fastest way to write tasks against wrong paths, so R1 says so explicitly and the plan extends the real files. Only verification earns a new leaf package (internal/security/bundlesig), on the toolsig argument: a pure fixture-testable wire contract an independent publisher must reproduce. - Spec 087 is specified but NOT implemented - no bundle package, no security .bundle config, no refresh ticker. FR-018 forbids inventing a second refresh lifecycle, and there is no first one to insert stages into. D11 therefore cuts scope at the network boundary: this feature delivers the file-drop path end to end and leaves fetch to 087. SC-003 (air-gapped install reaches full P1) is satisfied entirely by that path, which is the evidence the cut is right - but SC-007's fetch half is consequently not demonstrable by this feature alone, and that is recorded as needing the maintainer's assent rather than discovered by a task. - The eval gate cannot see the database at all. gateChecks() returns only the four built-in checks; scanner.BundleCheck, which production DOES append, is absent and scan-eval never loads a bundle. FR-016/SC-005 would pass today with the corpus entirely absent, so teaching the gate to score the bundle (D8) is a prerequisite for growing the corpus, not a parallel task. Decisions resolving the spec's two remaining clarification markers: publication goes to GitHub Releases on the corpus repo, keeping the artifact and the gate that must precede it in one workflow (D1); require_signed_bundle stays opt-in indefinitely, because the FR-009 ratchet already covers every install that has seen a signed bundle and a default flip would break only the air-gapped self-built-corpus persona (D2). Contracts carry the parts FR-003 and FR-001a explicitly refuse to leave to the implementation: a five-key sidecar with a total, reason-bearing rejection grammar and byte-level accept/reject fixtures, and an additive signatures[] section whose referential integrity is enforced in BOTH directions - the reverse direction (every gating rule named by exactly one signature) being the one that stops a gating rule shipping with no provenance, licence or eval evidence. * docs(specs): close cross-model review findings in the 101-tpa-db plan Cross-model review (opencode/gpt-5.6) of the plan, research and contracts, plus Go semantics checked empirically rather than asserted. Three P1s. **open(2) blocks on a FIFO before fstat can see it.** D5 specified O_NOFOLLOW + descriptor fstat + a context-cancelled reader, which looks airtight and is not: opening a writer-less FIFO blocks inside open itself, before fstat, before the reader goroutine, before any timeout exists to observe it. SC-011(c) — the never-written FIFO whose slot must be released within the budget — could not pass by construction. O_NONBLOCK is now part of the open and is cleared once fstat has confirmed a regular file, so an ordinary read keeps blocking semantics. **The last-known-good bundle needed content addressing, not just an fsync.** D6 said "persist LKG bytes, commit, publish" and claimed a pre-commit crash leaves the old (bundle, watermark) pair. False when the bytes land at a fixed filename: overwriting before the commit leaves NEW bytes under OLD state, the same split the requirement forbids, from the other side. LKG files are now tpa-lkg/<full-sha256>.json referenced by digest from the state record, so writing a new one never destroys the old; plus the directory fsync (a file fsync does not make its directory entry durable) and startup reconciliation of orphans. **Releasing the single-flight slot leaked an fd per refresh.** Found independently on both sides. Abandoned readers are now capped at one per path. Also corrected: - os.Root does NOT give the no-follow property on Windows — it blocks path escape but still follows symlinks resolving inside the root. Either handle-based opens with reparse-point controls, or the weaker guarantee written down. The candidate open is now build-tagged POSIX/Windows files, since syscall.O_NOFOLLOW does not compile there at all. - GitHub Release assets are mutable (delete and re-upload under the same name), so "immutable per-release asset names" asserted something the platform does not provide. Content-addressed names + a signed release index, cacheable at a stable URL with ETag and refresh jitter, since a fleet behind one NAT can exhaust the unauthenticated API limit in lockstep. - D2 is REOPENED rather than answered. "The ratchet does the work" describes installs that are already safe; the ratchet only arms after a first signed activation, so fresh installs are exactly the unprotected population. The recommendation is default-on with an explicit opt-out, but that breaks existing 086 unsigned droppers on upgrade, which is a maintainer's call, not mine. - The loader is now source-neutral from day one — Candidate{bytes, source} — so spec 087's fetch path cannot fork the verifier, which is the one component that must never exist twice. - SC-007's fetch clause moves to 087 in spec.md as an amendment task. A success criterion no fixture can exercise is a spec defect; assent does not make it testable. - Telemetry validates the whole tuple, not one direction, and carries schema_version + sequence instead of a per-release enum a shipped binary cannot know (which would classify every new corpus as "other" until the binary updates). Contract fixes, with the Go claims verified against Go 1.25 rather than assumed: - Rejection rows now have explicit first-match-wins precedence; they overlap ({"foo":1} is both missing-field and unknown-field) and without ordering two conforming implementations report different reasons. - Duplicate keys need their own scan. Measured: {"algorithm":"ed25519", "algorithm":"rsa"} decodes with err=nil and algorithm=="rsa", and DisallowUnknownFields does NOT catch it. On the field that pins the algorithm. - base64 must be canonical and strict. Measured: StdEncoding accepts embedded newlines and an 88-char string that does not round-trip; Strict() rejects it. - sequence dropped from the sidecar. Its stated rationale — detectable "before parsing" — was wrong, since detecting a mismatch requires parsing the manifest. It bought no pre-parse check and added an inconsistency state. - "Gating" is now mechanical (hard tier) rather than intent-prose: rules[] has no gating field, so the reverse integrity rule was unenforceable and a hard-tier rule could dodge eval evidence via a gating:false signature. - Every gating category must resolve to a registered categoryCheck, enforced at build. The contract explained that hazard and then did not check it. - last_rejection gains a detail field it was already referencing. * docs(specs): lock D2 — require_signed_bundle stays default-off Maintainer's call, against the recommendation to flip it on. Recorded with the trade that was put, so a future reader sees the reasoning rather than only the outcome: default-off keeps every existing 086 unsigned file-drop working across the upgrade, and the population a flip would protect (installs that have never activated a signed bundle) is the same one that has not adopted signing at all. Consequence now written into both documents: FR-007's coverage degradation and FR-020's surfacing of it are the ONLY thing between a fresh install and an attacker-supplied unsigned corpus. They stop being polish — shipping the loader without the degraded-state surfacing would ship the risk without the mitigation. Revisit trigger is SC-006 telemetry, which makes the question measurable.
1 parent af9927b commit f3edfa3

10 files changed

Lines changed: 1174 additions & 1 deletion

File tree

‎CLAUDE.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -158,6 +158,6 @@ tail -f ~/Library/Logs/mcpproxy/main.log # main log (macOS; Linux: ~/.mcpproxy/
158158
- **Windows installer**: [docs/github-actions-windows-wix-research.md](docs/github-actions-windows-wix-research.md). **Prerelease** (`next` branch + `v*-rc.*` tags, opt-in, off stable channels): [docs/prerelease-builds.md](docs/prerelease-builds.md).
159159

160160
## Recent Changes
161+
- 101-tpa-db: Go 1.25.5 module toolchain (`go.mod`), backend-only + stdlib only for the new work — `crypto/ed25519`, `crypto/sha256`, `syscall` (no ed25519 usage in the tree before this); existing bbolt (state), zap, Cobra. **No new dependencies.**
161162
- 098-tools-preflight: Added Go 1.24 module toolchain (repo builds with local Go 1.25) + existing only — chi (httpapi), bbolt (storage), Bleve (index), zap (logging), Cobra (CLI), swaggo/swag v2 (contract regen). **No new dependencies.**
162163
- 097-stored-scripts: Added Go 1.25 (os.Root/Root.ReadFile available — R1) + stdlib only (os.Root). **No new dependencies.**
163-
- 096-batched-call-tools: Added Go 1.24 module toolchain (repo builds with local Go 1.25) + existing only — goja (sandbox), mark3labs/mcp-go (tool surface), zap. **No new dependencies.**

‎ROADMAP.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -881,4 +881,5 @@ Legend: `shipped` ≥95% checked · `in-flight` 1–94% · `drafted` 0% · `—`
881881
| [098-tools-preflight](./specs/098-tools-preflight/) | `in-flight` | 26/33 (79%) |
882882
| [099-describe-check-mode](./specs/099-describe-check-mode/) | `in-flight` | 9/10 (90%) |
883883
| [100-prompt-rugpull-baseline](./specs/100-prompt-rugpull-baseline/) | — | — |
884+
| [101-tpa-db](./specs/101-tpa-db/) | — | — |
884885
| [102-schema-deferred](./specs/102-schema-deferred/) | — | — |
Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
# Contract: Bundle `signatures[]` Section
2+
3+
**Feature**: `101-tpa-db` · Resolves FR-001a.
4+
5+
## 1. Why this exists
6+
7+
FR-001 says the publishable artifact is "the existing compiled bundle". FR-012 says "each signature MUST carry provenance … and a redistributable license". **Both cannot hold today**, because the shipped `scanner-bundle.json` has nowhere to put either:
8+
9+
```text
10+
top level : bundle_version, generated_from, rules, schema_version, signature_count, skipped
11+
rules[] : category, confidence, detector, engine, flags, id, indicators,
12+
level, pattern, severity, target, type
13+
```
14+
15+
No provenance. No licence. No campaign identity — `signature_count` counts *rules*, so SC-004's "at least 25 signatures spanning at least 8 distinct classes" is not even expressible.
16+
17+
## 2. Shape
18+
19+
A new **additive** top-level array. A v0.1 loader ignores unknown top-level keys, so contract §4 forward-compat is preserved (FR-001a).
20+
21+
```json
22+
{
23+
"signatures": [
24+
{
25+
"id": "TPA-2026-0001",
26+
"category": "instruction_injection",
27+
"gating": true,
28+
"provenance": {
29+
"source": "https://invariantlabs.ai/blog/mcp-github-vulnerability",
30+
"published": "2026-01-14",
31+
"note": "hidden <IMPORTANT> block coaxing ~/.ssh/id_rsa"
32+
},
33+
"license": "CC-BY-4.0",
34+
"rule_ids": ["r-hidden-important-block", "r-hidden-html-comment"]
35+
}
36+
]
37+
}
38+
```
39+
40+
| Field | Type | Rules |
41+
|---|---|---|
42+
| `id` | string | `^TPA-[0-9]{4}-[0-9]{4}$`, unique across the bundle |
43+
| `category` | string | the campaign/technique class; must have a `categoryCheck` entry in the gate (FR-016a) |
44+
| `gating` | bool | `true` ⇒ must emit hard-tier signals and must ship eval evidence (FR-013/FR-014) |
45+
| `provenance.source` | URL | the public disclosure that motivated the signature |
46+
| `provenance.published` | `YYYY-MM-DD` | |
47+
| `provenance.note` | string | optional, human context |
48+
| `license` | SPDX id | from the redistributable allowlist below |
49+
| `rule_ids` | array | ≥1, each naming an existing `rules[]` entry |
50+
51+
**Licence allowlist** (redistributable, FR-012): `CC0-1.0`, `CC-BY-4.0`, `CC-BY-SA-4.0`, `MIT`, `Apache-2.0`. Anything else is refused at corpus build — including "no licence stated", which is the common case for a pattern lifted from a blog post and is precisely what FR-012 exists to stop shipping.
52+
53+
## 3. Relationship to `rules[]`
54+
55+
**Many-to-one, explicitly.** Several rules may implement one signature — a hidden-instruction class already needs distinct patterns for `<IMPORTANT>` blocks, HTML comments and zero-width runs. One rule belongs to at most one signature.
56+
57+
```text
58+
signatures[] ──1─────*── rules[]
59+
TPA id pattern
60+
```
61+
62+
## 4. Referential integrity — enforced at build, in both directions
63+
64+
A bundle failing any of these is **rejected whole**, exactly like any other contract violation:
65+
66+
1. every `signatures[]` entry names ≥1 rule id that exists in `rules[]`;
67+
2. every **hard-tier** rule is named by exactly one `gating: true` signature entry — not zero (an ungoverned gating rule has no provenance and no eval evidence) and not two (ambiguous campaign attribution).
68+
69+
**"Gating" must be mechanical, because `rules[]` has no such field.** Its keys are `category, confidence, detector, engine, flags, id, indicators, level, pattern, severity, target, type` — nothing that says "this rule gates approvals". Left as prose about intent, the check is unenforceable and a hard-tier rule can evade attribution entirely, or be attached to a `gating: false` signature and thereby dodge the FR-013 eval-evidence requirement while still driving the approval gate at runtime. The build MUST therefore derive it from `level` — a rule is gating **iff** it emits at the hard tier, which is exactly what FR-014 says gating means and exactly what the eval gate and the `scan`-mode approval gates score — and MUST reject any bundle where a hard-tier rule's owning signature has `gating: false`. `signatures[].gating` is then a declaration that is *checked* against the rules it names, not an independent switch;
70+
3. `id` values are unique;
71+
4. `license` is in the allowlist;
72+
5. every `gating: true` signature has its FR-013 eval pair in the frozen dataset — ≥1 labeled gated-malicious sample and ≥1 category-matched hard negative following the `hn_<attack_category>_*` naming the dataset validator enforces;
73+
6. every `gating: true` signature's `category` resolves to a **registered** `categoryCheck` entry in the gate. §5 below explains why an unmapped category is scored vacuously; that explanation is not enforcement, so this rule is what actually stops one shipping. A bundle naming a category the gate cannot enforce is rejected at build, not merely noted.
74+
75+
Direction 2 is the one a naive implementation drops, and it is the one that matters: checking only "signatures point at real rules" lets a gating rule ship with no campaign record at all, which is the state the corpus is in today. Rule 6 is the same failure one level up — a campaign record that exists but that the gate silently declines to score.
76+
77+
## 5. Interaction with the eval gate
78+
79+
`gating: true` is not self-certifying. FR-016a's `categoryCheck` map decides whether a category is *enforced*: `gatedCategory()` enforces a category only when its mapped check id is registered, so **a new class whose category is missing from that map is tallied and reported but excluded from `gatedMalicious`/`OverallRecall`** — its misses can never fail the gate. Adding a signature class therefore means adding its `categoryCheck` entry in the same change, or the coverage is decorative.
80+
81+
(Its false positives still count either way, since `FPRate` is computed over the whole hard-negative set regardless of category mapping. So an unmapped class can only ever hurt — never help — the gate outcome. That asymmetry is the tell.)
82+
83+
## 6. Counting for SC-004
84+
85+
- **signatures** = `len(signatures)` — must be ≥ 25
86+
- **classes** = distinct `category` — must be ≥ 8
87+
- every signature carries provenance + licence (rule 4 above)
88+
- every `gating: true` signature has its eval pair (rule 5)
89+
90+
Note `signature_count` (existing, top-level) counts **rules** and keeps that meaning. Renaming it would break the v0.1 line for no gain; SC-004 counts `signatures[]`.
Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
# Contract: Bundle Status Surface
2+
3+
**Feature**: `101-tpa-db` · Resolves FR-020.
4+
5+
The operator question this surface must answer is not "what version am I on" but **"am I current and verified, or stuck on last-known-good — and why?"** Today's `BundleInfo` cannot answer the second half at all.
6+
7+
## 1. Fields
8+
9+
Existing fields keep their meaning and their names. `Fingerprint` stays the 12-hex short form (it is load-bearing in current operator output and in `logBundle`); the full digest is **added beside it**, not substituted.
10+
11+
| Field | Existing | Purpose |
12+
|---|---|---|
13+
| `bundle_version`, `schema_version`, `source`, `generated_at`, `load_error` | ✅ | unchanged |
14+
| `runnable_rules`, `skipped_rules`, `declared_skipped` | ✅ | unchanged — un-evaluated coverage, never clean coverage |
15+
| `fingerprint` | ✅ | 12-hex short form, human-comparable |
16+
| `full_digest` | **new** | complete SHA-256; the **only** value rollback accepts (FR-010) |
17+
| `sequence` | **new** | decimal string, as published |
18+
| `authority` | **new** | `publisher` \| `operator` — which authority is serving |
19+
| `trust_epoch` | **new** | the epoch it was published under |
20+
| `signature_verified` | **new** | bool |
21+
| `staleness_days` | **new** | derived from `generated_at` |
22+
| `coverage_degraded` | **new** | bool |
23+
| `degraded_reason` | **new** | enum, §3 |
24+
| `last_rejection` | **new** | `{reason, detail, at}`, §2 — retained across a successful load so "what did it refuse, and when" survives. `detail` is a short human string carrying the part the enum cannot (which ceiling was exceeded and by how much, which key id was untrusted, which epoch was expected); it is never parsed, and `reason` is the only field anything branches on |
25+
26+
## 2. Rejection reasons
27+
28+
The vocabulary an operator reads to tell a tamper attempt from a stale epoch from their own misconfiguration:
29+
30+
| Reason | Means |
31+
|---|---|
32+
| `tamper` | signature did not verify over the raw bytes |
33+
| `downgrade` | sequence below this authority's watermark for this epoch, or equal sequence with a different fingerprint |
34+
| `stale_epoch_replay` | manifest epoch below the binary-trusted epoch — a validly-signed but superseded artifact |
35+
| `unsigned_refused_by_ratchet` | this install has activated a signed bundle before; unsigned drops are refused thereafter |
36+
| `authority_not_active` | verified against a real key of the **non-active** authority (FR-004b) |
37+
| `rollback_target_mismatch` | the loaded artifact's full digest ≠ the digest the operator named |
38+
| `gate_failure` | activation self-check (Spec 087 FR-008) rejected the candidate |
39+
| `ceiling_exceeded` | over one of FR-011's four limits — which limit, and the observed value, in `detail` |
40+
| `not_regular_file` | candidate was a FIFO/device/socket, or a symlink to one |
41+
| `read_timeout` | the 5 s read budget expired; the single-flight slot was released |
42+
| `sidecar_*`, `key_*`, `signature_invalid`, `sequence_mismatch` | the sidecar grammar — see [sidecar-format.md](sidecar-format.md) §3 |
43+
44+
**Every one of these keeps last-known-good serving.** There is no rejection path that leaves the scanner with an empty or partial rule set.
45+
46+
## 3. Degraded-coverage reasons
47+
48+
`coverage_degraded` suspends `scan`-mode auto-approval, routing changes to human review. It **must** be visible: a silently suspended auto-approval is indistinguishable from a quiet install, which is the failure this field exists to prevent.
49+
50+
| `degraded_reason` | Condition |
51+
|---|---|
52+
| `unsigned_external_active` | an unsigned file-dropped/fetched bundle is active (FR-007) |
53+
| `embedded_below_watermark` | the embedded default is serving and its sequence is below the **publisher** watermark (FR-009a a) |
54+
| `operator_bundle_missing` | the operator authority is active and its previously-activated external bundle is gone or invalid (FR-009a b) |
55+
| `operator_authority_unbaselined` | an operator-signed bundle is active under an authority configured without a baseline (FR-009b) |
56+
57+
**Not degraded** — the two cases a careless implementation gets wrong:
58+
59+
- an ordinary fresh install serving the embedded default with no retained watermark and no missing external bundle. The embedded default is binary-trusted; degrading here would ship every fresh install with `scan` mode silently disabled.
60+
- an embedded default **at or above** the publisher watermark after an external file was removed. It is demonstrably no older than what went missing — the ordinary "external file removed after a binary upgrade shipped a newer corpus" case.
61+
62+
## 4. Surfaces
63+
64+
Same field set everywhere; three renderings.
65+
66+
| Surface | Rendering |
67+
|---|---|
68+
| **CLI** (`mcpproxy security …`) | human table + `-o json`/`yaml`; the full digest shown in full (it is the rollback input) |
69+
| **REST** (security overview) | the JSON object above; `make swagger` regenerated |
70+
| **Web UI** (Security page) | short fingerprint with the full digest on demand; degraded state as a banner with its reason, never a silent flag |
71+
72+
The Web UI rule follows the house preference already established for status surfaces: **mark only the states that need action**, and always say *why* rather than showing a bare badge.
73+
74+
## 5. Activation history
75+
76+
`(authority, trust_epoch, sequence, full digest, generated_at, activated_at, source)` per activation, append-only, retained across restarts.
77+
78+
Exposed read-only. It exists because rollback is authorized by full digest and there is otherwise **no trustworthy way to learn the digest of the known-good artifact you want to revert to** — reading it from the candidate file would defeat the purpose, since the file is what the attacker controls.
79+
80+
Data-dir resident, so it carries the same caveat as the ratchet: it makes a legitimate rollback *possible*; it does not *authorize* one.

0 commit comments

Comments
 (0)