Skip to content

Commit 93486bc

Browse files
authored
Merge pull request DeusData#2189 from DeusData/fix/codeql-gate-fail-closed
fix(ci): make the code-scanning gate able to see alerts, and fail closed
2 parents 8d1f4b4 + 1ad52f5 commit 93486bc

5 files changed

Lines changed: 192 additions & 2 deletions

File tree

‎.github/workflows/_security.yml‎

Lines changed: 42 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,14 @@ jobs:
4141
codeql-gate:
4242
runs-on: ubuntu-latest
4343
timeout-minutes: 240
44+
# Without security-events:read this job inherits the workflow's
45+
# contents:read, the code-scanning alert API answers 403, and the gate
46+
# below reported "0 alerts" on every run it has ever made. actions:read is
47+
# what the CodeQL-run poll above needs for the same reason.
48+
permissions:
49+
contents: read
50+
security-events: read
51+
actions: read
4452
steps:
4553
- name: Wait for CodeQL on current commit (max 150 min)
4654
env:
@@ -77,11 +85,43 @@ jobs:
7785
env:
7886
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
7987
run: |
88+
# FAIL CLOSED. The previous form was `gh api ... 2>/dev/null || echo "0"`,
89+
# which looks defensive and is the opposite. gh writes the API error
90+
# BODY to stdout, so a 403 made ALERTS the string
91+
# {"message":"Resource not accessible by integration",...,"status":"403"}0
92+
# every `[` comparison then failed as non-integer, the `if` took its
93+
# false branch, and the step printed "CodeQL gate passed (0 alerts)"
94+
# and exited 0. That is exactly how a security gate should NEVER fail:
95+
# silently, in the direction of "everything is fine". An unreadable
96+
# alert list is now an error, not a zero.
97+
# NOTE: this runs in a command substitution, so it must `return`, not
98+
# `exit` -- an `exit` here would only end the subshell and let the
99+
# caller continue with an empty count, recreating the very bug this
100+
# replaces. The callers below check the status explicitly.
101+
count_open_alerts() {
102+
local body status
103+
body=$(gh api 'repos/${{ github.repository }}/code-scanning/alerts?state=open' \
104+
--jq 'length' 2>/dev/null)
105+
status=$?
106+
if [ $status -ne 0 ]; then
107+
echo "BLOCKED: cannot read code scanning alerts (gh exit $status)." >&2
108+
echo " Refusing to treat an unreadable alert list as zero." >&2
109+
echo " Most likely cause: the job lacks security-events:read." >&2
110+
return 1
111+
fi
112+
case "$body" in
113+
''|*[!0-9]*)
114+
echo "BLOCKED: alert count was not a number: ${body:-<empty>}" >&2
115+
return 1
116+
;;
117+
esac
118+
printf '%s' "$body"
119+
}
80120
echo "Waiting 60s for alert API to settle..."
81121
sleep 60
82-
ALERTS=$(gh api 'repos/${{ github.repository }}/code-scanning/alerts?state=open' --jq 'length' 2>/dev/null || echo "0")
122+
ALERTS=$(count_open_alerts) || exit 1
83123
sleep 15
84-
ALERTS2=$(gh api 'repos/${{ github.repository }}/code-scanning/alerts?state=open' --jq 'length' 2>/dev/null || echo "0")
124+
ALERTS2=$(count_open_alerts) || exit 1
85125
[ "$ALERTS" -lt "$ALERTS2" ] && ALERTS=$ALERTS2
86126
if [ "$ALERTS" -gt 0 ]; then
87127
echo "BLOCKED: $ALERTS open alert(s)"

‎.github/workflows/pr.yml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,13 @@ concurrency:
2222
jobs:
2323
security:
2424
uses: ./.github/workflows/_security.yml
25+
# The called job requests security-events:read to count code-scanning
26+
# alerts; a reusable workflow cannot request more than its caller grants,
27+
# so withholding this here fails the whole run at startup.
28+
permissions:
29+
contents: read
30+
security-events: read
31+
actions: read
2532
secrets: inherit
2633

2734
lint:

‎.github/workflows/release.yml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,13 @@ jobs:
4444
# ── Security (starts immediately, blocks only verify) ───────────
4545
security:
4646
uses: ./.github/workflows/_security.yml
47+
# The called job requests security-events:read to count code-scanning
48+
# alerts; a reusable workflow cannot request more than its caller grants,
49+
# so withholding this here fails the whole run at startup.
50+
permissions:
51+
contents: read
52+
security-events: read
53+
actions: read
4754
secrets: inherit
4855

4956
# ── 0. Preflight: refuse malformed dispatch inputs ──────────────

‎scripts/test.sh‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -218,6 +218,9 @@ bash "$ROOT/tests/test_windows_bundle_contract.sh"
218218
echo "=== Step 0e2: activation refusal diagnostic contract ==="
219219
bash "$ROOT/tests/test_activation_diagnostic_contract.sh"
220220

221+
echo "=== Step 0e3: security gate fail-closed contract ==="
222+
bash "$ROOT/tests/test_security_gate_fail_closed.sh"
223+
221224
echo "=== Step 0f: tree-sitter runtime Makefile dependencies ==="
222225
bash "$ROOT/tests/test_makefile_ts_runtime_dependencies.sh"
223226

Lines changed: 133 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,133 @@
1+
#!/usr/bin/env bash
2+
# The code-scanning gate must FAIL CLOSED.
3+
#
4+
# History. `codeql-gate` in .github/workflows/_security.yml counted open alerts
5+
# with:
6+
#
7+
# ALERTS=$(gh api '.../code-scanning/alerts?state=open' --jq 'length' 2>/dev/null || echo "0")
8+
#
9+
# That reads as defensive and is the exact opposite. The job declared no
10+
# `permissions:` block, so it inherited the workflow's `contents: read` and the
11+
# alert API answered 403. `gh` writes the API error BODY to stdout, so `ALERTS`
12+
# became the string
13+
#
14+
# {"message":"Resource not accessible by integration",...,"status":"403"}0
15+
#
16+
# every `[ ... -gt ... ]` then failed as a non-integer comparison, the `if` took
17+
# its false branch, and the step printed "CodeQL gate passed (0 alerts)" and
18+
# exited 0. Observed live in a GREEN run (34700514102 / job 103571322660), while
19+
# the repository had an open high-severity alert.
20+
#
21+
# A security gate that cannot read its input must stop the build, not wave it
22+
# through. This test pins both halves of that: the permission that makes the
23+
# read possible, and the fail-closed handling for when it still is not.
24+
set -euo pipefail
25+
26+
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
27+
28+
python3 - "$ROOT" <<'PY'
29+
from __future__ import annotations
30+
31+
import pathlib
32+
import re
33+
import sys
34+
35+
root = pathlib.Path(sys.argv[1])
36+
failures: list[str] = []
37+
38+
39+
def require(condition: bool, message: str) -> None:
40+
if not condition:
41+
failures.append(message)
42+
43+
44+
workflow = (root / ".github" / "workflows" / "_security.yml").read_text(encoding="utf-8")
45+
46+
# Isolate the codeql-gate job so a permission granted to some OTHER job cannot
47+
# satisfy this test.
48+
gate = re.search(r"\n codeql-gate:\n(?P<body>(?: .*\n|\n)*)", workflow)
49+
require(gate is not None, "_security.yml no longer defines a codeql-gate job")
50+
body = gate.group("body") if gate else ""
51+
52+
# 1. The read has to be possible at all.
53+
require(
54+
re.search(r"^ permissions:\s*$", body, re.M) is not None,
55+
"codeql-gate must declare its own permissions block -- inheriting the "
56+
"workflow default leaves it without security-events access and the alert "
57+
"API answers 403",
58+
)
59+
require(
60+
re.search(r"^ security-events:\s*read\s*$", body, re.M) is not None,
61+
"codeql-gate must request security-events: read, or the code-scanning "
62+
"alert query 403s and the gate cannot see any alert",
63+
)
64+
65+
# 2. The failure direction. `|| echo "0"` on the alert query is the specific
66+
# construct that turned an API error into a clean pass.
67+
require(
68+
re.search(r"code-scanning/alerts[^\n]*\|\|\s*echo", body) is None,
69+
"the alert count must not fall back to a literal on error -- an unreadable "
70+
"alert list is not zero alerts",
71+
)
72+
require(
73+
"BLOCKED: cannot read code scanning alerts" in body,
74+
"codeql-gate must fail with a named error when the alert API cannot be read",
75+
)
76+
require(
77+
re.search(r"alert count was not a number", body) is not None,
78+
"codeql-gate must reject a non-numeric alert count instead of comparing it",
79+
)
80+
81+
# 3. The count is produced in a command substitution, so a bare `exit` inside
82+
# the helper would end only the subshell and let the caller proceed with an
83+
# empty value -- the same class of silent pass. The callers must check.
84+
assignments = re.findall(r"^\s*ALERTS2?=\$\(count_open_alerts\)(.*)$", body, re.M)
85+
require(
86+
len(assignments) >= 2,
87+
"expected both alert samples to go through the checked helper",
88+
)
89+
require(
90+
all("|| exit 1" in tail for tail in assignments),
91+
"every count_open_alerts call must be followed by || exit 1 -- a failure "
92+
"inside a command substitution does not stop the caller on its own",
93+
)
94+
95+
# 4. Every caller must GRANT what the job requests. A reusable workflow cannot
96+
# request more permission than its caller holds: asking for
97+
# security-events:read from a caller that grants only contents:read does not
98+
# downgrade, it fails the entire run with a startup_failure before a single
99+
# job executes. Checking only the called workflow is how this test passed
100+
# while the PR pipeline never started.
101+
for caller_name in ("pr.yml", "release.yml"):
102+
caller = (root / ".github" / "workflows" / caller_name).read_text(encoding="utf-8")
103+
# The next sibling may be a comment line, not a key, so the lookahead has to
104+
# accept any 2-space-indented non-space -- matching only ` \w` silently
105+
# failed to find release.yml's job at all.
106+
call = re.search(
107+
r"\n security:\n(?P<body>(?: .*\n|\n)*?)(?=\n? \S|\Z)", caller
108+
)
109+
require(call is not None, f"{caller_name} no longer has a security: job")
110+
call_body = call.group("body") if call else ""
111+
require(
112+
"uses: ./.github/workflows/_security.yml" in call_body,
113+
f"{caller_name}'s security job must call _security.yml",
114+
)
115+
require(
116+
re.search(r"^ security-events:\s*read\s*$", call_body, re.M) is not None,
117+
f"{caller_name} must grant security-events: read to the security job -- "
118+
"a called workflow cannot request more than its caller grants, and the "
119+
"run fails at startup rather than degrading",
120+
)
121+
require(
122+
re.search(r"^ actions:\s*read\s*$", call_body, re.M) is not None,
123+
f"{caller_name} must grant actions: read to the security job",
124+
)
125+
126+
if failures:
127+
print("Security gate fail-closed contract FAILED:")
128+
for failure in failures:
129+
print(f" - {failure}")
130+
sys.exit(1)
131+
132+
print("Security gate fail-closed contract passed")
133+
PY

0 commit comments

Comments
 (0)