Skip to content

Commit bdb99d7

Browse files
committed
fix(release): stop re-scanning bytes VirusTotal has already scanned
The verify pass submitted every extracted object, selected executables included, on the stated grounds that "VirusTotal is content-addressed, so identical bytes return the analysis it already holds instead of re-running 70+ engines". That is measurably false. On v0.10.5 all EIGHT re-submissions produced a NEW analysis - same VirusTotal file-id, timestamp 47 minutes later: candidate: file-id=2c00f485... ts=1786795957 (12:12:37Z) verify : file-id=2c00f485... ts=1786798758 (12:59:18Z) Re-analysing identical bytes re-rolls a probabilistic classifier, and Microsoft's ML engine answered differently within that hour, in BOTH directions: 82750cd1 (linux-amd64) microsoft-ml -> clean 6d3c5be6 (darwin-arm64) clean -> microsoft-ml The published notes are generated from the candidate scan, so v0.10.5 shipped a table calling linux-amd64 flagged when VirusTotal had it clean, and darwin-arm64 clean when VirusTotal was reporting Trojan:Script/Wacatac.B!ml. Every hash in that table links to the page that contradicted it. Corrected in place after publication; this removes the cause. The second scan proved nothing the first did not. Identity is settled by hash before this step runs: verify-release-selection.py reconciles every published container to the selected bytes, and checksums.txt binds the same digests publicly. A re-scan adds no assurance - only another roll. What still gets scanned is exactly what the candidate pass never saw: install.sh, install.ps1, LICENSE, THIRD_PARTY_NOTICES.md, the MCPB manifest.json and the unpacked UI assets. install.sh and install.ps1 are the highest-consequence non-executable bytes we publish - users pipe them straight into a shell - and that coverage is untouched. Measured on the v0.10.5 object set: 16 objects in, 8 withheld, 8 still scanned. The withheld set is recorded as evidence (cbm-virustotal-withheld-v1) naming each sha256 and pointing at virustotal-candidate-results.tsv, so the published evidence still accounts for every shipped object. Fails closed three ways, each with an actionable message: no object matches a selected sha (the containers do not carry the recorded bytes), everything is withheld (the surface scan would be a no-op), or the selection names no shas at all. The zero-match grep is wrapped rather than left to pipefail, because a guard that aborts silently is not a guard - found by testing the guards rather than assuming them. Also drops 8 VirusTotal submissions per release. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
1 parent 3162f09 commit bdb99d7

2 files changed

Lines changed: 118 additions & 4 deletions

File tree

‎.github/workflows/release.yml‎

Lines changed: 17 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -316,10 +316,23 @@ jobs:
316316
# install.sh and install.ps1 are the highest-consequence non-executable
317317
# bytes we publish: users pipe them straight into a shell.
318318
#
319-
# The selected executables are re-submitted too. That is deliberate and
320-
# nearly free: VirusTotal is content-addressed, so identical bytes return
321-
# the analysis it already holds instead of re-running 70+ engines — the
322-
# same property that made the analysis-id equality check untenable.
319+
# The selected executables are NOT re-submitted. They were scanned as
320+
# candidates, and verify-release-selection.py has just proven every
321+
# published container carries exactly those bytes — identity is settled by
322+
# hash, so a second scan adds no assurance.
323+
#
324+
# It also is not free, contrary to what this comment used to claim.
325+
# Measured on v0.10.5: all eight re-submissions produced a NEW analysis
326+
# (same file-id, timestamp 47 minutes later), and Microsoft's ML engine
327+
# answered differently for two of them within that hour — in opposite
328+
# directions. The release notes then cited one scan while the links showed
329+
# the other. See scripts/ci/exclude-rescanned-selected-objects.sh.
330+
- name: Withhold already-scanned selected executables
331+
run: |
332+
bash scripts/ci/exclude-rescanned-selected-objects.sh \
333+
binaries/objects "$RUNNER_TEMP/release-selection.tsv" \
334+
binaries/virustotal-withheld.tsv
335+
323336
- name: VirusTotal scan of every extracted release object
324337
uses: crazy-max/ghaction-virustotal@936d8c5c00afe97d3d9a1af26d017cfdf26800a2 # v5.0.0
325338
id: virustotal
Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
#!/usr/bin/env bash
2+
# Withhold the selected executables from the full-surface VirusTotal pass.
3+
#
4+
# WHY (measured, not assumed). The verify pass used to submit every extracted
5+
# object, selected executables included, on the stated grounds that "VirusTotal
6+
# is content-addressed, so identical bytes return the analysis it already holds
7+
# instead of re-running 70+ engines". That is not what happens. On the v0.10.5
8+
# release all EIGHT re-submissions produced a NEW analysis: same VirusTotal
9+
# file-id, a timestamp 47 minutes later.
10+
#
11+
# candidate: file-id=2c00f485... ts=1786795957 (12:12:37Z)
12+
# verify : file-id=2c00f485... ts=1786798758 (12:59:18Z)
13+
#
14+
# Re-analysing identical bytes re-rolls a probabilistic classifier, and it
15+
# answered differently in the same hour, in both directions:
16+
#
17+
# 82750cd1 (linux-amd64) microsoft-ml -> clean
18+
# 6d3c5be6 (darwin-arm64) clean -> microsoft-ml
19+
#
20+
# The published release notes then cited the candidate verdict while the linked
21+
# page showed the verify verdict, so the table said "clean" for a binary
22+
# VirusTotal was flagging and vice versa. Nothing about the binaries differed:
23+
# same sha256 in both scans.
24+
#
25+
# The second scan also proved nothing the first did not. Identity is already
26+
# established by hash: verify-release-selection.py reconciles every published
27+
# container to the selected bytes before this step runs, and checksums.txt binds
28+
# the same digests publicly. A re-scan adds no assurance and one more roll.
29+
#
30+
# What still gets scanned is everything the candidate pass never saw: install.sh,
31+
# install.ps1, LICENSE, THIRD_PARTY_NOTICES.md, the MCPB manifest.json and the
32+
# unpacked UI assets. install.sh and install.ps1 are the highest-consequence
33+
# non-executable bytes we publish - users pipe them straight into a shell - and
34+
# that coverage is untouched.
35+
#
36+
# Fails closed: if nothing matches, the selection binding is broken; if nothing
37+
# is left, the surface scan would silently become a no-op.
38+
#
39+
# Usage: exclude-rescanned-selected-objects.sh <objects-dir> <release-selection.tsv> [manifest-out]
40+
set -euo pipefail
41+
42+
OBJECTS_DIR="${1:?usage: exclude-rescanned-selected-objects.sh <objects-dir> <selection.tsv> [manifest-out]}"
43+
SELECTION="${2:?usage: exclude-rescanned-selected-objects.sh <objects-dir> <selection.tsv> [manifest-out]}"
44+
MANIFEST="${3:-}"
45+
46+
test -d "$OBJECTS_DIR" || { echo "error: no objects directory: $OBJECTS_DIR" >&2; exit 1; }
47+
test -s "$SELECTION" || { echo "error: no selection evidence: $SELECTION" >&2; exit 1; }
48+
49+
head -n 1 "$SELECTION" | grep -qx '# cbm-release-selection-v1' || {
50+
echo "error: wrong evidence marker in $SELECTION" >&2; exit 1; }
51+
52+
# selected_sha256 is the 4th column; skip the '#' metadata block and the header.
53+
selected="$(mktemp)"
54+
withheld="$(mktemp)"
55+
trap 'rm -f "$selected" "$withheld"' EXIT
56+
# `|| true` on the grep: with `set -o pipefail` a zero-match grep would abort the
57+
# script here, before the explicit check below could say why. A guard that exits
58+
# silently is not a guard.
59+
awk -F '\t' '/^#/ {next} $1=="target" {next} {print $4}' "$SELECTION" \
60+
| { grep -E '^[0-9a-f]{64}$' || true; } | sort -u > "$selected"
61+
if [ ! -s "$selected" ]; then
62+
echo "error: no selected sha256 values in $SELECTION — the selection" >&2
63+
echo " evidence names no shipped bytes, so nothing can be matched." >&2
64+
exit 1
65+
fi
66+
67+
kept=0
68+
for f in "$OBJECTS_DIR"/*; do
69+
[ -f "$f" ] || continue
70+
sha="$(sha256sum "$f" | awk '{print $1}')"
71+
if grep -qx "$sha" "$selected"; then
72+
printf '%s\t%s\n' "$sha" "$(basename "$f")" >> "$withheld"
73+
rm -f -- "$f"
74+
else
75+
kept=$((kept + 1))
76+
fi
77+
done
78+
79+
count="$(wc -l < "$withheld" | tr -d ' ')"
80+
if [ "$count" -eq 0 ]; then
81+
echo "error: no extracted object matched a selected sha256 — the release" >&2
82+
echo " containers do not carry the bytes the selection recorded." >&2
83+
exit 1
84+
fi
85+
if [ "$kept" -eq 0 ]; then
86+
echo "error: every extracted object was withheld; the full-surface scan" >&2
87+
echo " would cover nothing." >&2
88+
exit 1
89+
fi
90+
91+
if [ -n "$MANIFEST" ]; then
92+
{
93+
echo "# cbm-virustotal-withheld-v1"
94+
echo "# reason=already-scanned-as-candidate"
95+
echo "# evidence=virustotal-candidate-results.tsv"
96+
printf 'sha256\tobject\n'
97+
cat "$withheld"
98+
} > "$MANIFEST"
99+
fi
100+
101+
echo "withheld $count already-scanned selected executable(s); $kept object(s) remain for the surface scan"

0 commit comments

Comments
 (0)