Skip to content

Commit fec1022

Browse files
authored
Merge pull request DeusData#1619 from DeusData/fix/vt-notes-shipped-only
fix(ci): release notes report the shipped binary, not every candidate
2 parents c0bd4bb + 5f82732 commit fec1022

4 files changed

Lines changed: 46 additions & 22 deletions

File tree

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -794,7 +794,7 @@ internal/cbm/ Vendored tree-sitter grammars (158 languages) + AST extrac
794794

795795
Every release is verified through a multi-layer pipeline before publication:
796796

797-
- **VirusTotal** — all 24 executable candidates (unstripped, debug-stripped and stripped) across the eight release products are scanned before smoke/soak (clean is preferred and only the single-Microsoft `!ml` tolerance documented in [SECURITY.md](SECURITY.md#our-release-policy) may pass; the number of engines that returned a decisive result is recorded as evidence but is VirusTotal fleet availability, not a pass condition). The selected executable is packaged without changing its SHA-256, and release notes link every candidate result and the selection evidence. Every distinct object extracted from the shipped containers — `install.sh`, `install.ps1`, `LICENSE`, `THIRD_PARTY_NOTICES.md`, the MCPB `manifest.json` and the unpacked UI assets — is then scanned as well, so the full published surface is covered, not just the executables.
797+
- **VirusTotal** — all 24 executable candidates (unstripped, debug-stripped and stripped) across the eight release products are scanned before smoke/soak (clean is preferred and only the single-Microsoft `!ml` tolerance documented in [SECURITY.md](SECURITY.md#our-release-policy) may pass; the number of engines that returned a decisive result is recorded as evidence but is VirusTotal fleet availability, not a pass condition). The selected executable is packaged without changing its SHA-256, release notes link the verdict for the exact bytes shipped, and the full per-candidate evidence is published alongside the release as TSVs for anyone auditing the selection. Every distinct object extracted from the shipped containers — `install.sh`, `install.ps1`, `LICENSE`, `THIRD_PARTY_NOTICES.md`, the MCPB `manifest.json` and the unpacked UI assets — is then scanned as well, so the full published surface is covered, not just the executables.
798798
- **SLSA Level 3** — cryptographic build provenance generated by the trusted GitHub Actions build workflow; verify with `gh attestation verify <file> --repo DeusData/codebase-memory-mcp --signer-workflow DeusData/codebase-memory-mcp/.github/workflows/_build.yml`
799799
- **Sigstore cosign** — keyless signatures on all artifacts; bundles included in every release
800800
- **SHA-256 checksums** — `checksums.txt` published with every release; verified by both install scripts before extraction

‎SECURITY.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -126,7 +126,7 @@ Releases are created as **drafts** (invisible to users) and only published after
126126
2. **Sigstore cosign signing** — keyless digital signatures verifiable by anyone
127127
3. **SBOM** — Software Bill of Materials (SPDX) listing all vendored dependencies
128128
4. **SHA-256 checksums** — published with every release
129-
5. **VirusTotal scanning** — three behaviourally identical executable candidates — unstripped, debug-stripped and stripped — are derived from one linker output for every release product and scanned before smoke/soak. Zero malicious and zero suspicious is preferred; the only tolerated result is exactly one Microsoft malicious label ending in `!ml`, which is disclosed. The selected executable is packaged without changing its SHA-256. After packaging, every distinct object extracted from the 14 shipped containers is scanned under the same policy — `install.sh`, `install.ps1`, `LICENSE`, `THIRD_PARTY_NOTICES.md`, the MCPB `manifest.json` and the unpacked UI assets — so the covered surface is everything we publish, not only the executables. Release notes link every candidate result and the selection evidence.
129+
5. **VirusTotal scanning** — three behaviourally identical executable candidates — unstripped, debug-stripped and stripped — are derived from one linker output for every release product and scanned before smoke/soak. Zero malicious and zero suspicious is preferred; the only tolerated result is exactly one Microsoft malicious label ending in `!ml`, which is disclosed. The selected executable is packaged without changing its SHA-256. After packaging, every distinct object extracted from the 14 shipped containers is scanned under the same policy — `install.sh`, `install.ps1`, `LICENSE`, `THIRD_PARTY_NOTICES.md`, the MCPB `manifest.json` and the unpacked UI assets — so the covered surface is everything we publish, not only the executables. Release notes link the verdict for the exact bytes shipped; the per-candidate results and the selection table are published as release assets for independent audit.
130130
6. **OpenSSF Scorecard** — repository security health score
131131

132132
Scope of the SLSA claim: this is a build provenance claim for release

‎scripts/ci/append-vt-notes.sh‎

Lines changed: 27 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -211,40 +211,52 @@ asset_base = (
211211
)
212212
213213
214-
def verdict_cell(target: str, variant: str) -> str:
215-
result = results_by_path[candidate_by_pair[target, variant]["scan_path"]]
214+
# Release notes report the SHIPPED binary and nothing else.
215+
#
216+
# Several candidates per product are scanned so the selector has an alternative
217+
# when an opaque classifier flags one of them, but a reader installing cbm cares
218+
# about the bytes they receive, not about the ones we discarded. The rejected
219+
# candidates' verdicts stay in the published evidence TSVs for anyone auditing
220+
# the selection, and they matter to US in development as a signal; they are
221+
# noise in a changelog.
222+
def shipped_result(target: str) -> dict:
223+
selection = selection_by_target[target]
224+
variant = selection["selected_variant"]
225+
return results_by_path[candidate_by_pair[target, variant]["scan_path"]]
226+
227+
228+
def verdict_cell(target: str) -> str:
229+
result = shipped_result(target)
216230
label = "clean" if result["policy_classification"] == "clean" else "Microsoft `!ml`"
217231
return f"[{label}]({result['virustotal_url']})"
218232
219233
220-
ml_count = sum(
221-
result["policy_classification"] == "microsoft-ml" for result in results
222-
)
223-
engine_counts = [int(result["completed_engines"]) for result in results]
234+
shipped = [shipped_result(target) for target in TARGETS]
235+
ml_count = sum(result["policy_classification"] == "microsoft-ml" for result in shipped)
236+
engine_counts = [int(result["completed_engines"]) for result in shipped]
224237
engine_range = str(min(engine_counts)) if min(engine_counts) == max(engine_counts) else f"{min(engine_counts)}–{max(engine_counts)}"
225238
section = [
226239
START,
227240
"## Security Verification",
228241
"",
229242
(
230-
f"Before smoke and soak testing, VirusTotal completed **{len(TARGETS) * len(VARIANTS)} executable scans**: "
231-
"stripped and unstripped candidates for each of the **8 release products**. "
232-
f"Every scan had at least 50 decisive engines (observed range: {engine_range})."
243+
f"Every binary published below was scanned by VirusTotal before smoke and soak "
244+
f"testing, and the verdict for the exact shipped bytes is linked per product "
245+
f"(decisive engines: {engine_range})."
233246
),
234247
(
235-
"All candidates were clean."
248+
"Every shipped binary was clean."
236249
if ml_count == 0
237-
else f"**{ml_count} candidate(s)** had only the documented single Microsoft machine-learning `!ml` result; no other decisive engine reported malicious or suspicious."
250+
else f"**{ml_count} shipped binary/binaries** carried only the documented single Microsoft machine-learning `!ml` result; no other decisive engine reported malicious or suspicious."
238251
),
239252
"",
240-
"| Product | Stripped candidate | Unstripped candidate | Shipped |",
241-
"|---|---|---|---|",
253+
"| Product | Shipped binary | VirusTotal verdict |",
254+
"|---|---|---|",
242255
]
243256
for target in TARGETS:
244257
selection = selection_by_target[target]
245258
section.append(
246-
f"| `{target}` | {verdict_cell(target, 'stripped')} | "
247-
f"{verdict_cell(target, 'unstripped')} | `{selection['selected_variant']}` (`{selection['selected_sha256']}`) |"
259+
f"| `{target}` | `{selection['selected_sha256']}` | {verdict_cell(target)} |"
248260
)
249261
section.extend(
250262
[

‎tests/test_vt_release_notes_contract.sh‎

Lines changed: 17 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,12 @@ classes = {(target, variant): "clean" for target in targets for variant in varia
6565
classes["linux-amd64", "stripped"] = "microsoft-ml"
6666
classes["darwin-arm64", "stripped"] = "microsoft-ml"
6767
classes["darwin-arm64", "unstripped"] = "microsoft-ml"
68+
# windows-amd64 draws the tolerated verdict on ALL THREE candidates, so the
69+
# SHIPPED binary carries it and the notes must disclose exactly that one. The
70+
# two targets above keep a clean sibling, so their flagged candidates are
71+
# rejected and must NOT appear in the notes at all.
72+
for _variant in variants:
73+
classes["windows-amd64", _variant] = "microsoft-ml"
6874
candidates = []
6975
results = []
7076
by_pair = {}
@@ -190,12 +196,18 @@ run_notes "$FIX/current.md" "$FIX/first.md"
190196
grep -q 'Intro text.' "$FIX/first.md" || fail "content before section was lost"
191197
grep -q 'Outro text.' "$FIX/first.md" || fail "content after section was lost"
192198
! grep -q 'stale data' "$FIX/first.md" || fail "stale section was appended"
193-
grep -q '24 executable scans' "$FIX/first.md" || fail "candidate scan scope missing"
194-
grep -q '8 release products' "$FIX/first.md" || fail "product scope missing"
195-
grep -q '3 candidate(s).*Microsoft' "$FIX/first.md" || fail "tolerated findings not disclosed"
199+
# Release notes report ONLY the shipped bytes. Rejected candidates stay in the
200+
# published evidence TSVs; they are development signal, not changelog content.
201+
grep -q 'verdict for the exact shipped bytes is linked per product' "$FIX/first.md" || \
202+
fail "shipped-binary scan scope missing"
203+
[ "$(grep -c '^| `' "$FIX/first.md")" = "8" ] || fail "product scope missing: expected one row per release product"
204+
grep -q '1 shipped binary.*Microsoft' "$FIX/first.md" || fail "tolerated finding on a shipped binary not disclosed"
205+
grep -q 'candidate(s)' "$FIX/first.md" && fail "notes must not count rejected candidates"
196206
grep -q 'no other decisive engine reported malicious or suspicious' "$FIX/first.md" || fail "non-decisive engines are overstated as clean"
197207
! grep -q 'every other engine was clean' "$FIX/first.md" || fail "non-decisive engines are incorrectly described as clean"
198-
grep -q '| Product | Stripped candidate | Unstripped candidate | Shipped |' "$FIX/first.md" || fail "candidate table missing"
208+
grep -q '| Product | Shipped binary | VirusTotal verdict |' "$FIX/first.md" || fail "shipped-binary table missing"
209+
grep -q 'Stripped candidate' "$FIX/first.md" && \
210+
fail "release notes must not enumerate rejected candidates"
199211
grep -q '| `linux-amd64` |' "$FIX/first.md" || fail "Linux product row missing"
200212
grep -q '| `windows-arm64` |' "$FIX/first.md" || fail "Windows product row missing"
201213
grep -q 'archive containers were not redundantly submitted' "$FIX/first.md" || fail "no-rescan boundary missing"
@@ -228,4 +240,4 @@ if run_notes "$FIX/reversed.md" "$FIX/reversed-capture.md"; then
228240
fail "reversed verification markers must fail closed"
229241
fi
230242

231-
echo 'PASS: release notes reuse all 24 candidate verdicts, disclose selection, and avoid a duplicate scan'
243+
echo 'PASS: release notes report the shipped binary only, disclose selection, and avoid a duplicate scan'

0 commit comments

Comments
 (0)