Skip to content

Commit 7b4533b

Browse files
committed
fix(ci): finish the three-candidate wiring and restore full-surface VT scanning
Two things, both found by the dry-run and by sweeping what it exposed. 1. The three-candidate change was incomplete. The dry-run failed at stage-release-candidates.py, which carries its OWN copy of the transform validation that the previous commit only fixed in the selector: stage-release-candidates: candidate linux-amd64/debug-stripped has invalid transform: 'strip-debug' Sweeping for that assumption found it in six places, not two: stage-release-candidates.py, select-release-candidates.py, verify-release-selection.py, append-vt-notes.sh and two contract-test fixtures — as hardcoded 16s, `len(TARGETS) * 2`, two-entry VARIANTS tuples and two-key truth tables. All are now derived from len(VARIANTS). Field prefixes needed care: the variant NAME keeps its hyphen because it is the on-disk directory, while the evidence columns use underscores, so `debug-stripped` reads `debug_stripped_sha256`. Every lookup now goes through an explicit FIELD_KEY map instead of interpolating the variant directly. The selection contract test now covers the truth table EXHAUSTIVELY: three variants x two tolerated classifications is exactly eight combinations, and there are exactly eight targets, so every case is exercised once. 2. Full-surface VirusTotal scanning is restored. This PR had moved scanning upstream to the candidates and deleted the post-package pass, which silently narrowed coverage from everything we ship to executables only. The 42 runtime files across the 14 containers — install.sh, install.ps1, LICENSE, THIRD_PARTY_NOTICES.md, the MCPB manifest.json and the unpacked UI assets — were still extracted, structurally verified and strings-audited, but no longer scanned at all. install.sh and install.ps1 are the highest-consequence non-executable bytes we publish; users pipe them straight into a shell. The verify job scans every extracted object again, under the same policy. Re-submitting the selected executables alongside them is close to free because VirusTotal is content-addressed and answers for identical bytes from its own record — the same property that made the analysis-id equality check untenable two commits ago. The gate-chain contract asserted the opposite ("duplicate post-package VirusTotal path remains"). That assertion is inverted: the pass is required, and it is not a duplicate, since it covers a strictly larger set. README and SECURITY.md updated from "archive containers are checksummed rather than redundantly rescanned" to state the full covered surface. All five release/VT contract tests pass. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
1 parent 98d7dba commit 7b4533b

10 files changed

Lines changed: 199 additions & 90 deletions

‎.github/workflows/release.yml‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -298,6 +298,60 @@ jobs:
298298
scripts/security-strings.sh "$f"
299299
done
300300
301+
# Full-surface VirusTotal pass over the SHIPPED containers.
302+
#
303+
# The candidate scan upstream covers executables only. Everything else we
304+
# actually ship — install.sh, install.ps1, LICENSE, THIRD_PARTY_NOTICES.md,
305+
# the MCPB manifest.json and the unpacked UI assets, 42 runtime files
306+
# across 14 containers — is extracted here and gets scanned here. Dropping
307+
# it would have narrowed the promise made in README/SECURITY.md, and
308+
# install.sh and install.ps1 are the highest-consequence non-executable
309+
# bytes we publish: users pipe them straight into a shell.
310+
#
311+
# The selected executables are re-submitted too. That is deliberate and
312+
# nearly free: VirusTotal is content-addressed, so identical bytes return
313+
# the analysis it already holds instead of re-running 70+ engines — the
314+
# same property that made the analysis-id equality check untenable.
315+
- name: VirusTotal scan of every extracted release object
316+
uses: crazy-max/ghaction-virustotal@936d8c5c00afe97d3d9a1af26d017cfdf26800a2 # v5.0.0
317+
id: virustotal
318+
with:
319+
vt_api_key: ${{ secrets.VIRUS_TOTAL_SCANNER_API_KEY }}
320+
files: binaries/objects/*
321+
request_rate: 4
322+
323+
# Same policy as the candidate gate: a single Microsoft `!ml` is tolerated
324+
# and disclosed, anything else blocks. Engine count is evidence, not a gate.
325+
- name: Wait for VirusTotal results
326+
env:
327+
VT_API_KEY: ${{ secrets.VIRUS_TOTAL_SCANNER_API_KEY }}
328+
VT_ANALYSIS: ${{ steps.virustotal.outputs.analysis }}
329+
VT_EXPECTED_SCAN_SET: binaries/scan-set.tsv
330+
VT_ASSOCIATIONS: binaries/associations.tsv
331+
VT_RESULTS_PATH: binaries/vt-results.tsv
332+
MIN_ENGINES: 50
333+
run: scripts/ci/check-virustotal.sh
334+
335+
- name: Preserve extracted-object VirusTotal evidence
336+
if: ${{ always() }}
337+
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
338+
with:
339+
name: virustotal-evidence-release-${{ github.run_id }}
340+
path: |
341+
binaries/associations.tsv
342+
binaries/scan-set.tsv
343+
binaries/vt-results.tsv
344+
if-no-files-found: warn
345+
346+
- name: Publish durable public VirusTotal evidence
347+
env:
348+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
349+
VERSION: ${{ inputs.version }}
350+
VT_EXPECTED_SCAN_SET: binaries/scan-set.tsv
351+
VT_ASSOCIATIONS: binaries/associations.tsv
352+
VT_RESULTS_PATH: binaries/vt-results.tsv
353+
run: bash scripts/ci/publish-vt-evidence.sh
354+
301355
# Candidate bytes were scanned before smoke/soak and the verifier above
302356
# proved every packaged executable has the selected candidate SHA-256.
303357
# Re-submitting the same executable bytes here would add wait time but no

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -794,7 +794,7 @@ internal/cbm/ Vendored tree-sitter grammars (158 languages) + AST extrac
794794

795795
Every release is verified through a multi-layer pipeline before publication:
796796

797-
- **VirusTotal** — all 24 executable candidates (unstripped, debug-stripped and stripped) across the eight release products are scanned before smoke/soak (clean is preferred and only the single-Microsoft `!ml` tolerance documented in [SECURITY.md](SECURITY.md#our-release-policy) may pass; the number of engines that returned a decisive result is recorded as evidence but is VirusTotal fleet availability, not a pass condition). The selected executable is packaged without changing its SHA-256, and release notes link every candidate result and the selection evidence. Archive containers are checksummed, not redundantly resubmitted.
797+
- **VirusTotal** — all 24 executable candidates (unstripped, debug-stripped and stripped) across the eight release products are scanned before smoke/soak (clean is preferred and only the single-Microsoft `!ml` tolerance documented in [SECURITY.md](SECURITY.md#our-release-policy) may pass; the number of engines that returned a decisive result is recorded as evidence but is VirusTotal fleet availability, not a pass condition). The selected executable is packaged without changing its SHA-256, and release notes link every candidate result and the selection evidence. Every distinct object extracted from the shipped containers — `install.sh`, `install.ps1`, `LICENSE`, `THIRD_PARTY_NOTICES.md`, the MCPB `manifest.json` and the unpacked UI assets — is then scanned as well, so the full published surface is covered, not just the executables.
798798
- **SLSA Level 3** — cryptographic build provenance generated by the trusted GitHub Actions build workflow; verify with `gh attestation verify <file> --repo DeusData/codebase-memory-mcp --signer-workflow DeusData/codebase-memory-mcp/.github/workflows/_build.yml`
799799
- **Sigstore cosign** — keyless signatures on all artifacts; bundles included in every release
800800
- **SHA-256 checksums** — `checksums.txt` published with every release; verified by both install scripts before extraction

‎SECURITY.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -126,7 +126,7 @@ Releases are created as **drafts** (invisible to users) and only published after
126126
2. **Sigstore cosign signing** — keyless digital signatures verifiable by anyone
127127
3. **SBOM** — Software Bill of Materials (SPDX) listing all vendored dependencies
128128
4. **SHA-256 checksums** — published with every release
129-
5. **VirusTotal scanning** — three behaviourally identical executable candidates — unstripped, debug-stripped and stripped — are derived from one linker output for every release product and scanned before smoke/soak. Zero malicious and zero suspicious is preferred; the only tolerated result is exactly one Microsoft malicious label ending in `!ml`, which is disclosed. The selected executable is packaged without changing its SHA-256; archive containers are checksummed rather than redundantly rescanned. Release notes link every candidate result and the selection evidence.
129+
5. **VirusTotal scanning** — three behaviourally identical executable candidates — unstripped, debug-stripped and stripped — are derived from one linker output for every release product and scanned before smoke/soak. Zero malicious and zero suspicious is preferred; the only tolerated result is exactly one Microsoft malicious label ending in `!ml`, which is disclosed. The selected executable is packaged without changing its SHA-256. After packaging, every distinct object extracted from the 14 shipped containers is scanned under the same policy — `install.sh`, `install.ps1`, `LICENSE`, `THIRD_PARTY_NOTICES.md`, the MCPB `manifest.json` and the unpacked UI assets — so the covered surface is everything we publish, not only the executables. Release notes link every candidate result and the selection evidence.
130130
6. **OpenSSF Scorecard** — repository security health score
131131

132132
Scope of the SLSA claim: this is a build provenance claim for release

‎scripts/ci/append-vt-notes.sh‎

Lines changed: 25 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,8 @@ TARGETS = (
3838
"windows-amd64",
3939
"windows-arm64",
4040
)
41-
VARIANTS = ("unstripped", "stripped")
41+
VARIANTS = ("unstripped", "debug-stripped", "stripped")
42+
FIELD_KEY = {variant: variant.replace("-", "_") for variant in VARIANTS}
4243
CANDIDATE_FIELDS = (
4344
"target", "variant", "relative_path", "source_sha256", "pre_sign_sha256",
4445
"sha256", "size", "format", "architecture", "linkage", "transform",
@@ -54,7 +55,11 @@ SELECTION_FIELDS = (
5455
"target", "selected_variant", "selected_path", "selected_sha256",
5556
"selected_size", "decision", "unstripped_sha256", "unstripped_scan_path",
5657
"unstripped_classification", "unstripped_analysis_id",
57-
"unstripped_virustotal_url", "stripped_sha256", "stripped_scan_path",
58+
"unstripped_virustotal_url",
59+
"debug_stripped_sha256", "debug_stripped_scan_path",
60+
"debug_stripped_classification", "debug_stripped_analysis_id",
61+
"debug_stripped_virustotal_url",
62+
"stripped_sha256", "stripped_scan_path",
5863
"stripped_classification", "stripped_analysis_id", "stripped_virustotal_url",
5964
)
6065
SHA256 = re.compile(r"[0-9a-f]{64}\Z")
@@ -110,20 +115,22 @@ selection_meta, selections = read_tsv(
110115
selection_path, "cbm-release-selection-v1", SELECTION_FIELDS
111116
)
112117
expected_pairs = [(target, variant) for target in TARGETS for variant in VARIANTS]
113-
if candidate_meta != {"targets": "8", "candidates": "16"}:
114-
fail("candidate metadata does not bind the 8-target/16-candidate matrix")
118+
if candidate_meta != {"targets": str(len(TARGETS)), "candidates": str(len(TARGETS) * len(VARIANTS))}:
119+
fail("candidate metadata does not bind the canonical target/variant matrix")
115120
if [(row["target"], row["variant"]) for row in candidates] != expected_pairs:
116121
fail("candidate rows are not the canonical target/variant matrix")
117-
if result_meta.get("scan_objects") != "16" or result_meta.get("associations") != "16":
118-
fail("VirusTotal results do not cover all 16 candidates")
119-
if selection_meta != {"policy": "virustotal-v2", "targets": "8", "candidates": "16"}:
122+
expected_scans = str(len(TARGETS) * len(VARIANTS))
123+
if result_meta.get("scan_objects") != expected_scans or result_meta.get("associations") != expected_scans:
124+
fail("VirusTotal results do not cover every candidate")
125+
if selection_meta != {"policy": "virustotal-v2", "targets": str(len(TARGETS)), "candidates": str(len(TARGETS) * len(VARIANTS))}:
120126
fail("selection was not produced by the scanned release policy")
121127
if [row["target"] for row in selections] != list(TARGETS):
122128
fail("selection rows are not the canonical target matrix")
123129
124130
candidate_by_pair = {(row["target"], row["variant"]): row for row in candidates}
125131
candidate_by_path = {row["scan_path"]: row for row in candidates}
126-
if len(candidate_by_pair) != 16 or len(candidate_by_path) != 16:
132+
expected_candidates = len(TARGETS) * len(VARIANTS)
133+
if len(candidate_by_pair) != expected_candidates or len(candidate_by_path) != expected_candidates:
127134
fail("candidate evidence contains duplicates")
128135
129136
results_by_path: dict[str, dict[str, str]] = {}
@@ -175,9 +182,10 @@ for selection in selections:
175182
target = selection["target"]
176183
pair = {variant: candidate_by_pair[target, variant] for variant in VARIANTS}
177184
pair_results = {variant: results_by_path[pair[variant]["scan_path"]] for variant in VARIANTS}
178-
stripped_class = pair_results["stripped"]["policy_classification"]
179-
unstripped_class = pair_results["unstripped"]["policy_classification"]
180-
expected_variant = "stripped" if stripped_class == "clean" or unstripped_class != "clean" else "unstripped"
185+
classes = {v: pair_results[v]["policy_classification"] for v in VARIANTS}
186+
order = ("stripped", "debug-stripped", "unstripped")
187+
clean = [v for v in order if classes[v] == "clean"]
188+
expected_variant = clean[0] if clean else "stripped"
181189
selected = pair[expected_variant]
182190
if (
183191
selection["selected_variant"] != expected_variant
@@ -188,11 +196,11 @@ for selection in selections:
188196
for variant in VARIANTS:
189197
result = pair_results[variant]
190198
if (
191-
selection[f"{variant}_sha256"] != pair[variant]["sha256"]
192-
or selection[f"{variant}_scan_path"] != pair[variant]["scan_path"]
193-
or selection[f"{variant}_classification"] != result["policy_classification"]
194-
or selection[f"{variant}_analysis_id"] != result["analysis_id"]
195-
or selection[f"{variant}_virustotal_url"] != result["virustotal_url"]
199+
selection[f"{FIELD_KEY[variant]}_sha256"] != pair[variant]["sha256"]
200+
or selection[f"{FIELD_KEY[variant]}_scan_path"] != pair[variant]["scan_path"]
201+
or selection[f"{FIELD_KEY[variant]}_classification"] != result["policy_classification"]
202+
or selection[f"{FIELD_KEY[variant]}_analysis_id"] != result["analysis_id"]
203+
or selection[f"{FIELD_KEY[variant]}_virustotal_url"] != result["virustotal_url"]
196204
):
197205
fail(f"selection evidence is not bound to both candidate verdicts: {target}")
198206
selection_by_target[target] = selection
@@ -219,7 +227,7 @@ section = [
219227
"## Security Verification",
220228
"",
221229
(
222-
"Before smoke and soak testing, VirusTotal completed **16 executable scans**: "
230+
f"Before smoke and soak testing, VirusTotal completed **{len(TARGETS) * len(VARIANTS)} executable scans**: "
223231
"stripped and unstripped candidates for each of the **8 release products**. "
224232
f"Every scan had at least 50 decisive engines (observed range: {engine_range})."
225233
),

‎scripts/ci/select-release-candidates.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -253,7 +253,7 @@ def load_candidates(
253253
marker="cbm-release-candidates-v1",
254254
fields=CANDIDATE_FIELDS,
255255
)
256-
if metadata != {"targets": str(len(TARGETS)), "candidates": str(len(TARGETS) * 2)}:
256+
if metadata != {"targets": str(len(TARGETS)), "candidates": str(len(TARGETS) * len(VARIANTS))}:
257257
raise ContractError("candidate metadata does not bind the exact canonical matrix")
258258
expected_pairs = [(target, variant) for target in TARGETS for variant in VARIANTS]
259259
if [(row["target"], row["variant"]) for row in rows] != expected_pairs:

‎scripts/ci/stage-release-candidates.py‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -194,7 +194,11 @@ def validate_row_shape(row: dict[str, str], *, target: str, variant: str) -> pat
194194
"format": file_format,
195195
"architecture": architecture,
196196
"linkage": linkage,
197-
"transform": "copy" if variant == "unstripped" else "strip",
197+
"transform": {
198+
"unstripped": "copy",
199+
"debug-stripped": "strip-debug",
200+
"stripped": "strip",
201+
}[variant],
198202
"signature": "adhoc-verified" if target.startswith("darwin-") else "not-applicable",
199203
"pair_verification": "same-linker-output-v1",
200204
}

‎scripts/ci/verify-release-selection.py‎

Lines changed: 30 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,8 @@ class ContractError(Exception):
3131
"windows-amd64",
3232
"windows-arm64",
3333
)
34-
VARIANTS = ("unstripped", "stripped")
34+
VARIANTS = ("unstripped", "debug-stripped", "stripped")
35+
FIELD_KEY = {variant: variant.replace("-", "_") for variant in VARIANTS}
3536
SELECTION_FIELDS = (
3637
"target",
3738
"selected_variant",
@@ -44,6 +45,11 @@ class ContractError(Exception):
4445
"unstripped_classification",
4546
"unstripped_analysis_id",
4647
"unstripped_virustotal_url",
48+
"debug_stripped_sha256",
49+
"debug_stripped_scan_path",
50+
"debug_stripped_classification",
51+
"debug_stripped_analysis_id",
52+
"debug_stripped_virustotal_url",
4753
"stripped_sha256",
4854
"stripped_scan_path",
4955
"stripped_classification",
@@ -163,7 +169,7 @@ def load_selection(
163169
target = row["target"]
164170
binary = "codebase-memory-mcp.exe" if target.startswith("windows-") else "codebase-memory-mcp"
165171
classifications = {
166-
variant: row[f"{variant}_classification"] for variant in VARIANTS
172+
variant: row[f"{FIELD_KEY[variant]}_classification"] for variant in VARIANTS
167173
}
168174
if metadata["policy"] == "virustotal-v2":
169175
if any(
@@ -172,35 +178,41 @@ def load_selection(
172178
):
173179
raise ContractError(f"release selection contains a hard classification: {target}")
174180
for variant in VARIANTS:
175-
sha256 = row[f"{variant}_sha256"]
176-
analysis_id = row[f"{variant}_analysis_id"]
181+
sha256 = row[f"{FIELD_KEY[variant]}_sha256"]
182+
analysis_id = row[f"{FIELD_KEY[variant]}_analysis_id"]
177183
expected_url = f"https://www.virustotal.com/gui/file/{sha256}/detection"
178184
if ANALYSIS_ID_RE.fullmatch(analysis_id) is None:
179185
raise ContractError(f"release selection lacks VT analysis evidence: {target}")
180186
if analysis_id in analysis_ids:
181187
raise ContractError("release selection reuses one VT analysis id")
182188
analysis_ids.add(analysis_id)
183-
if row[f"{variant}_virustotal_url"] != expected_url:
189+
if row[f"{FIELD_KEY[variant]}_virustotal_url"] != expected_url:
184190
raise ContractError(f"release selection has unbound VT URL: {target}")
185-
stripped_class = classifications["stripped"]
186-
unstripped_class = classifications["unstripped"]
187-
if stripped_class == "clean":
188-
expected_variant = "stripped"
189-
expected_decision = "stripped-preferred"
190-
elif unstripped_class == "clean":
191-
expected_variant = "unstripped"
192-
expected_decision = "unstripped-clean-after-stripped-microsoft-ml"
191+
# Re-derived here independently of the selector: smallest artifact
192+
# first, first CLEAN one wins, and only if every candidate drew the
193+
# tolerated verdict do we ship the smallest flagged one.
194+
order = ("stripped", "debug-stripped", "unstripped")
195+
clean = [v for v in order if classifications[v] == "clean"]
196+
if clean:
197+
expected_variant = clean[0]
198+
if expected_variant == "stripped":
199+
expected_decision = "stripped-preferred"
200+
else:
201+
others = "-".join(
202+
f"{v}:{classifications[v]}" for v in order if v != expected_variant
203+
)
204+
expected_decision = f"{expected_variant}-clean-after-{others}"
193205
else:
194206
expected_variant = "stripped"
195-
expected_decision = "stripped-both-microsoft-ml"
207+
expected_decision = "stripped-all-candidates-microsoft-ml"
196208
else:
197209
if any(
198210
classification != "unscanned-dry-run"
199211
for classification in classifications.values()
200212
):
201213
raise ContractError(f"dry-run selection disguises unscanned evidence: {target}")
202214
if any(
203-
row[f"{variant}_{suffix}"]
215+
row[f"{FIELD_KEY[variant]}_{suffix}"]
204216
for variant in VARIANTS
205217
for suffix in ("analysis_id", "virustotal_url")
206218
):
@@ -217,14 +229,14 @@ def load_selection(
217229
raise ContractError(f"release selection has unsafe selected path: {target}")
218230
if (
219231
SHA256_RE.fullmatch(row["selected_sha256"]) is None
220-
or row["selected_sha256"] != row[f"{selected_variant}_sha256"]
232+
or row["selected_sha256"] != row[f"{FIELD_KEY[selected_variant]}_sha256"]
221233
or not row["selected_size"].isdigit()
222234
or not 0 < int(row["selected_size"]) <= MAX_MEMBER_BYTES
223235
):
224236
raise ContractError(f"release selection has invalid selected content: {target}")
225237
for variant in VARIANTS:
226-
sha256 = row[f"{variant}_sha256"]
227-
scan_path = pathlib.PurePosixPath(row[f"{variant}_scan_path"])
238+
sha256 = row[f"{FIELD_KEY[variant]}_sha256"]
239+
scan_path = pathlib.PurePosixPath(row[f"{FIELD_KEY[variant]}_scan_path"])
228240
if (
229241
SHA256_RE.fullmatch(sha256) is None
230242
or scan_path.as_posix() != f"objects/{sha256}"

0 commit comments

Comments
 (0)