[Dumbris]

This PR is the third and last piece of the tpa-db epic. It implements Spec 101 User Story 3 together with Spec 087 User Stories 1 and 2. The TPA signature database now refreshes itself, can optionally be fetched as a signed artifact, proves a new corpus is no worse before it activates, shows how fresh it is, and re-checks servers that were already approved when new signatures arrive.

Related #938

What changes

Refresh lifecycle (Spec 087 US1). internal/security/scanner/tpa_refresh.go runs one cycle per tpa_refresh_interval (default 24h, minimum 1h, jittered). Each cycle has two legs:

• The file leg re-reads tpa_bundle_path, so a dropped bundle activates without a reload or a restart.
• The optional fetch leg runs after it.

Both legs feed the same source-neutral evaluateCandidate pipeline that US1 built. The verifier is not forked (Spec 101 D11).

Concurrency and scheduling:

• A manual refresh, a timer tick and a config nudge that happen together join one cycle.
• Network I/O runs outside bundleRT.mu.
• Scheduling is one deadline held under the refresher's lock.
• The refresher is joined in Runtime.Close before storage stops.

Signed fetch (Spec 087 US2, Spec 101 D1). The fetch is opt-in and off by default.

• Keys: tpa_fetch_enabled, tpa_fetch_url, and the env override MCPPROXY_TPA_FETCH_ENABLED. The env override is never persisted and cannot change the URL.
• Channel shapes: a GitHub Releases API releases/latest URL selects scanner-bundle-<seq>.json plus .sig. Any other URL is the "direct" shape: <url> plus <url>.sig.
• Signing: fetched pairs are signed-only even with require_signed_bundle off. They are verified against the US1 authority trust set.
• Transport:
  • https only, with plain http allowed only for a loopback host.
  • Redirects are capped and can never downgrade from https.
  • Egress is checked at dial time. Link-local and metadata addresses are always refused.
  • Bytes, the cycle and response headers are bounded.
  • ETag/If-None-Match is used, and Retry-After and rate-limit backoff are honoured.
  • Only the URL host ever reaches logs, status or errors.
• Outcomes vs rejections: network outcomes (fetch_unavailable, not_modified, channel_invalid, superseded, disabled) are kept apart from last_rejection and never move the watermark.
• Persistence: a fetched bundle survives reloads and restarts. Turning the fetch off keeps it serving.

Activation self-check (087 FR-008). tpa_selfcheck.go runs the CI gate decision against every candidate, layering it on the built-in checks:

• recall ≥ 0.90 over gated categories,
• hard-negative FP ≤ 0.05,
• fail closed when the corpus is vacuous.

It uses an embedded byte copy of the eval corpus (bundled/selfcheck-corpus.json). A candidate that regresses is refused as gate_failure, and last-known-good keeps serving.

Freshness you can see (Spec 101 FR-020). BundleInfo gains last_checked_at, last_activated_at, stale/stale_after_days and refresh{…} (trigger, outcome, reason, fetch leg state, next check, consecutive failures), plus reprotect{…}. These show on:

• GET /api/v1/security/bundle and /security/overview (swagger regenerated),
• mcpproxy security bundle,
• the Web UI signature panel.

mcpproxy doctor warns in three cases:

• the active database is older than tpa_stale_after_days (default 30),
• it is stuck on last-known-good after a refusal or after ≥3 failed fetches,
• the fetch is enabled but cannot run (no URL, invalid URL, or no trusted signer).

Manual refresh. mcpproxy security bundle refresh calls POST /api/v1/security/bundle/refresh (admin only). It runs one joined cycle with both legs. reload still never touches the network.

Re-protection (Spec 101 FR-019). Every activation or rollback fires an observer that runs a sweep off the hot path:

• Leg (i) refreshes the held scan evidence of pending and changed tools, quarantined servers included.
• Leg (ii) runs a cached-contract informational scan of approved toolsets on non-quarantined servers.
  • It uses no upstream connection and is pinned to the activated bundle snapshot.
  • It is settled by job ID, makes one job per (server, digest), and is gated by auto_baseline_scan.

RT-02 holds: nothing is approved, unquarantined, revoked or quarantined. A completion marker in BBolt drives a startup catch-up, on stdio too.

Telemetry schema 16. Five integer counters are added to tpa_scanner inside the closed anonymity contract:

• bundle_refresh_cycles
• bundle_refresh_activations
• bundle_refresh_rejections
• bundle_refresh_unavailable
• bundle_reprotect_hits

source=fetch is mapped. An operator-signed fetch still reports other with no sequence.

Specs and docs.

• SC-007's fetch clause moves from Spec 101 to Spec 087 as SC-009.
• Spec 087 is marked implemented by Spec 101 US3. Its internal/security/bundle package was never built.
• The US3 tasks (T300–T396) and the 2026-10-21 re-validation delta are in specs/101-tpa-db/tasks.md.
• Docs updated:
  • docs/features/tpa-bundle-signing.md (refresh and fetch),
  • docs/features/security-quarantine.md (re-protection),
  • docs/features/tool-scanner.md (self-check),
  • docs/configuration.md (keys),
  • docs/cli/security-commands.md,
  • docs/features/telemetry.md,
  • quickstart, CLAUDE.md, roadmap.

No new dependencies.

Assumptions and decisions (no maintainer input requested; please sign off)

All of these are recorded in specs/101-tpa-db/tasks.md (Δ15–Δ31, B1–B23).

• There is no working default fetch URL. The repository smart-mcp-proxy/tpa-db does not exist yet, and the publisher key set is still empty (US1 A1). The fetch therefore works today only with tpa_active_root: operator plus tpa_operator_signers. With the default publisher root it makes zero requests and doctor reports no_trusted_signer.

• Channel shape (Δ22). Per-sequence asset names are only selection hints. Integrity comes from the signed pair, the per-authority watermark and the deny-list. Research D1's signed release index is deferred.

• The self-check is a ported copy of the gate (Δ17). It is not shared code: cmd/scan-eval is package main and belongs to US2. Parity tests guard it:

  • corpus bytes,
  • an AST comparison with gate.go that accepts both the gateChecks() and gateChecks(bundle) shapes,
  • per-entry hard flags,
  • an optional CLI differential.

  This deviates from 101 FR-016a(c). Moving the gate into one shared package is a follow-up.

• The self-check skips last-known-good restores (B13). It runs for normal activations and rollbacks only, so an upgrade cannot deactivate a bundle that already passed.

• A download whose settings changed mid-flight is discarded as superseded (Δ30). It is not activated under the old settings.

• Bundle status stays off /api/v1/info (Δ19). /info is readable by agent tokens. Release status, which Spec 079 already provides, stays separate.

• Turning the fetch off keeps the fetched bundle serving (B9). rollback --embedded is the explicit way back.

• Re-protection leg (ii) depends on auto_baseline_scan (B17). While that switch is off the sweep reports suspended and resumes when the switch comes back on.

• There is no new attention or review-queue kind (B18). Findings show in the scan summary, the badge, the overview, the activity log and the per-tool scan_verdict on the server's review payload.

• Malformed refresh keys never block boot. A malformed tpa_refresh_interval, tpa_stale_after_days, tpa_fetch_enabled or tpa_fetch_url in a hand-edited file is neutralised in place, with a value-free diagnostic, and the fetch is forced off. API writes stay strict.

Merge order with US2 (#1702)

TestSelfCheckCorpusByteParity pins bundled/selfcheck-corpus.json to specs/065-evaluation-foundation/datasets/detect_corpus_v1.json. US2 grows that corpus, so whichever PR merges second runs:

cp specs/065-evaluation-foundation/datasets/detect_corpus_v1.json internal/security/scanner/bundled/selfcheck-corpus.json

The self-check tests were made independent of the corpus size, and they pass on a trial merge with US2 plus that copy. On US2's corpus a self-check takes about 3.3 s per candidate without -race, and a full -race run of the scanner package takes about 20 minutes, so give it -timeout 40m.

Verified on a live instance

These ran on head aba38112c with go build. The setup was:

• port 18741 with a scratch data_dir and config,
• tpa_active_root: operator, tpa_operator_baseline: 1, and a throwaway key from tpa-db keygen,
• bundles made from the embedded corpus plus manifest fields, signed with tpa-db sign,
• seq 3 and later carry one extra canary rule TPA-2099-0001, which matches the text zebra-canary-7731,
• two stdio fixture servers whose tool descriptions contain that text: fx-approved (trusted, tool approved) and fx-held (quarantined, tool pending).

The output below is trimmed with jq.

# 1. start, signed seq 2 at tpa_bundle_path
{"source":"file","sequence":"2","signature_verified":true,"refresh":{"last_trigger":"startup","next_check_at":"…09:00:28"}}

# 2. drop seq 3 (adds TPA-2099-0001), no reload: the first timer cycle activates it
{"source":"file","sequence":"3","refresh":{"last_trigger":"timer","last_outcome":"activated"},"reprotect":{"state":"idle","servers_scanned":1,"servers_with_hits":1}}
# re-protection: the approved tool is flagged and stays approved; the held tool's evidence picks up the new signature
{"server":"fx-approved","quarantined":false,"tools":[{"name":"lookup_weather","approval_status":"approved","scan_verdict":"dangerous"}]}
{"server":"fx-held","quarantined":true,"tools":[{"name":"fetch_notes","approval_status":"pending","held_reason":"scan_findings","held_signals":["tpa.TPA-2099-0001.hidden_instruction"]}]}

# 3. tampered seq 4 + `mcpproxy security bundle refresh`: last-known-good keeps serving
{"sequence":"3","refresh":{"last_trigger":"manual","last_outcome":"rejected"},"last_rejection":{"reason":"tamper"}}
$ mcpproxy doctor
  • TPA signature bundle: a newer candidate was refused (tamper); still serving last-known-good (see `mcpproxy security bundle`)

# 4. valid seq 4 + refresh
{"source":"file","sequence":"4","refresh":{"last_trigger":"manual","last_outcome":"activated"}}

# 5. fetch OFF (loopback mirror serving signed seq 5 on :18742): refresh + reload
{"source":"file","sequence":"4","refresh":{"last_outcome":"disabled","last_reason":"fetch_off"}}
mirror requests with fetch off: 0

# 6. tpa_fetch_enabled: true via config hot reload, then refresh
mirror requests after reload, before refresh: 0
    Source:          fetch (http://127.0.0.1)
    Refresh:         activated, next check …
    Fetch host:      127.0.0.1
    Sequence:        5
    Signature:       verified
mirror log: "GET /seq5.json" 200, "GET /seq5.json.sig" 200
log lines containing the URL path: 0

# 7. restart: the fetched bundle persists (the older file is refused once as downgrade, a quiet reason)
{"source":"fetch","path":"http://127.0.0.1","sequence":"5","signature_verified":true}

# 8. signed seq 6 with generated_at 2026-09-10, reload, doctor
    STALE:           42 days old (threshold 30 days) - run `mcpproxy security bundle refresh`, or check that the refresh source is reachable
  • TPA signature bundle is stale: 42 days old (stale after 30 days); run `mcpproxy security bundle refresh` or check the refresh source

The Web UI Review page on the same instance shows the new rows in the signature panel: last checked, last activated, refresh, fetch host, and the stale message. ./scripts/run-web-smoke.sh with the fixture upstream gave 127 passed, 3 skipped.

Review record

• Internal:

  • An Opus judge over the full diff raised 10 defects, all real and all fixed.
  • A 6-dimension review workflow with 3 adversarial refuters per finding produced 14 findings. 5 survived and were fixed.

• Cross-model: opencode GPT-6.1 Sol (variant high), file-scoped chunks, 6 rounds plus one small final chunk. Every reported finding was checked against the code before fixing. All were real and fixed test-first, with each test shown to fail on the old code.

  Round	Findings	Chunks
  1	19	9 of 10
  2	11	5 of 9
  3	6	4 of 5
  4	4	5
  5	4	4 of 5
  6	0	all clean

  Every non-test file in the diff is in a chunk whose last verdict is CLEAN at the current code. After round 4 the refresh timer was redesigned around a single deadline (escalated to Opus), and the config normaliser was changed to splice values in place instead of re-marshalling.

CI notes

• CodeQL: red on 4 go/allocation-size-overflow alerts in files this PR does not change:

  • internal/security/scanner/tpa_trust_anchors.go
  • internal/server/mcp_max_result_size.go
  • internal/server/profile_tool.go

  The two profile_tool.go:413 alerts are already open on main (fix(runtime): serve indexed tools when StateView per-server cache is empty (MCP-2083) #635, chore(deps): bump the go-dependencies group with 5 updates #646). CodeQL says it re-attributes alerts when a diff is large. CodeQL is not a required check.

• Telemetry:

  • Checked: the receiver has only lower-bound schema gates, passes tpa_scanner through, and stores the raw payload, so schema 16 is accepted. No dashboard panel reads the new counters.
  • Not checked: the live dashboard is behind Cloudflare Access OTP and was not loaded.

Out of scope / follow-ups

• Publication half of FR-017:
  • creating the corpus repository and its gate-then-sign-then-publish workflow,
  • shipping the first publisher key,
  • a signed release index.
• A shared gate package used by both cmd/scan-eval and the self-check, after US2 lands.
• An attention or review-queue kind for "an approved tool now matches signature X". This needs the Spec 109 parity matrix.
• The receiver-side SC-006 freshness join.

🤖 Generated with Claude Code