Skip to content

Commit 47ab375

Browse files
committed
fix(security): a bearer token gated on a substring, not a host
CodeQL raised py/incomplete-url-substring-sanitization on scripts/check_action_refs.py, and unlike the three alerts recorded in SECURITY.md it was right about the mechanism, not just the smell. if token and "api.github.com" in url: request.add_header("Authorization", f"Bearer {token}") That substring is present in `https://elsewhere.example/?next=api.github.com`, in `https://api.github.com.evil.example/`, and in `https://evil.example/api.github.com/`. What hangs off the test is whether a GitHub token is attached, so the loose form hands a credential to any host that cares to mention the name. Not exploitable today -- every URL in that file is built from a literal host -- which is exactly how a check like this survives review until the day someone makes the host a parameter. Now compares urlsplit(url).hostname, and four tests hold it there: the real API host still gets the token (or the rest would prove nothing), three lookalikes get none, and pypi.org -- the other host this same function reaches -- gets none either. SECURITY.md renumbered to the current alert ids and now distinguishes the three analyser false positives from this one, which was a defect and was fixed rather than explained away.
1 parent 18ce982 commit 47ab375

3 files changed

Lines changed: 90 additions & 8 deletions

File tree

SECURITY.md

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -51,8 +51,15 @@ Out of scope:
5151
## Static-analysis findings that are open on purpose
5252

5353
CodeQL (`security-extended`) runs on every push. Three alerts are open and are
54-
not defects. They are recorded here rather than only in a dismissal box, so the
55-
reasoning is reviewable and can be re-checked when the code changes.
54+
not defects. A fourth was raised and **was** a defect -- a bearer token attached
55+
after an `"api.github.com" in url` substring test, which is true of any host that
56+
mentions the name. That one was fixed in `scripts/check_action_refs.py` rather
57+
than explained away, and `tests/test_secrets_do_not_leak.py` now proves three
58+
lookalike hosts receive no credential.
59+
60+
The three below are different in kind. They are recorded here rather than only in
61+
a dismissal box, so the reasoning is reviewable and can be re-checked when the
62+
code changes.
5663

5764
All three come from CodeQL's **name-based** sensitive-data classifier: it treats
5865
identifiers spelled `key`, `secret` or `password` as sensitive sources and then
@@ -63,8 +70,8 @@ plants real-shaped secrets and asserts on the actual bytes.
6370

6471
| Alert | Query | Location | Why it fires, and why it is wrong |
6572
|---|---|---|---|
66-
| #4, #5 | `py/clear-text-logging-sensitive-data` | `tooltrace/cli/main.py` (`_emit`) | A verification key reaches `a2a.report()`, whose result is printed. Inside, the key is used only as the first argument to `hmac.new` and `hmac.compare_digest`; the returned structure holds signature *states*, algorithm names and prose. Two tests drive both output paths with a real key and assert it appears in neither stdout nor stderr. |
67-
| #3 | `py/clear-text-storage-sensitive-data` | `tooltrace/security/redaction.py` (`write_record`) | The record contains `residual_secret_classes` -- the **labels** of secret patterns that still match after sanitisation, such as `aws_access_key`, never the matched text. `Finding` documented that in a comment and nothing checked it; a test now plants an email, a card number and an AWS key and asserts none appears in the report, the record, or either file written to disk. |
73+
| #2, #3 | `py/clear-text-logging-sensitive-data` | `tooltrace/cli/main.py` (`_emit`) | A verification key reaches `a2a.report()`, whose result is printed. Inside, the key is used only as the first argument to `hmac.new` and `hmac.compare_digest`; the returned structure holds signature *states*, algorithm names and prose. Two tests drive both output paths with a real key and assert it appears in neither stdout nor stderr. |
74+
| #1 | `py/clear-text-storage-sensitive-data` | `tooltrace/security/redaction.py` (`write_record`) | The record contains `residual_secret_classes` -- the **labels** of secret patterns that still match after sanitisation, such as `aws_access_key`, never the matched text. `Finding` documented that in a comment and nothing checked it; a test now plants an email, a card number and an AWS key and asserts none appears in the report, the record, or either file written to disk. |
6875

6976
These are not suppressed in source. A `# codeql[...]` comment on `_emit` would
7077
blanket every command that prints anything, which is exactly the kind of

scripts/check_action_refs.py

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,7 @@
3838
import re
3939
import sys
4040
import urllib.error
41+
import urllib.parse
4142
import urllib.request
4243
from pathlib import Path
4344

@@ -69,8 +70,14 @@ def _get_json(url: str) -> object | None:
6970
url,
7071
headers={"Accept": "application/vnd.github+json", "User-Agent": "check-action-refs"},
7172
)
73+
# Compare the parsed host, never a substring. `"api.github.com" in url` is
74+
# true of `https://elsewhere.example/?x=api.github.com`, and what hangs off
75+
# this test is whether a credential is attached -- so the loose form is a
76+
# token-disclosure bug waiting for the day the URL stops being a literal.
77+
# CodeQL flagged exactly this, and unlike the three alerts recorded in
78+
# SECURITY.md it was right about the mechanism.
7279
token = os.environ.get("GITHUB_TOKEN") or os.environ.get("GH_TOKEN")
73-
if token and "api.github.com" in url:
80+
if token and urllib.parse.urlsplit(url).hostname == "api.github.com":
7481
request.add_header("Authorization", f"Bearer {token}")
7582
try:
7683
# Fixed https hosts, built below from literals.

tests/test_secrets_do_not_leak.py

Lines changed: 71 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,18 @@
22
33
Six CodeQL alerts landed on this repository, all `high`, three of them naming
44
this file's subject: clear-text logging in `cli/main.py` and clear-text storage
5-
in `security/redaction.py`. Every one turned out to be a false positive for the
6-
vulnerability it claimed -- the analyser cannot see through `report()` or
7-
`redaction_report()` and assumed taint propagated.
5+
in `security/redaction.py`. Those three were false positives for the
6+
vulnerability they claimed -- the analyser cannot see through `report()` or
7+
`redaction_report()` and assumed taint propagated. They are recorded, with
8+
reasons, in SECURITY.md.
9+
10+
A seventh was not a false positive, and the last section here is about it:
11+
`scripts/check_action_refs.py` decided whether to attach a bearer token with
12+
`"api.github.com" in url`. That substring is present in
13+
`https://elsewhere.example/?next=api.github.com`, so the loose test would have
14+
handed a GitHub token to any host that cared to mention the name. Not
15+
exploitable while every URL was built from a literal -- which is exactly how a
16+
check like that survives review until the day it isn't.
817
918
"Assumed" is the operative word, in both directions. Nothing here *proved* the
1019
key never reached the output, or that the redaction record never quoted what it
@@ -188,3 +197,62 @@ def test_the_record_still_reports_that_it_found_something(tmp_path: Path) -> Non
188197
)
189198
record = redaction_report(bundle)
190199
assert record["pii_findings"], "nothing was detected, so the leak check proves nothing"
200+
201+
202+
# --- a token goes to one host, and host means host ----------------------------
203+
204+
205+
def _captured_request(url: str, monkeypatch) -> object:
206+
"""Run `_get_json` against `url`, returning the Request it would have sent."""
207+
import importlib.util
208+
import sys as _sys
209+
import urllib.error
210+
import urllib.request
211+
212+
spec = importlib.util.spec_from_file_location(
213+
"ttb_check_action_refs_test",
214+
Path(__file__).resolve().parent.parent / "scripts" / "check_action_refs.py",
215+
)
216+
assert spec and spec.loader
217+
module = importlib.util.module_from_spec(spec)
218+
_sys.modules[spec.name] = module
219+
spec.loader.exec_module(module)
220+
221+
seen: list[object] = []
222+
223+
def fake_urlopen(request, *a, **k):
224+
seen.append(request)
225+
raise urllib.error.URLError("not sent")
226+
227+
monkeypatch.setenv("GITHUB_TOKEN", SECRET)
228+
monkeypatch.setattr(urllib.request, "urlopen", fake_urlopen)
229+
with pytest.raises(urllib.error.URLError):
230+
module._get_json(url)
231+
return seen[0]
232+
233+
234+
def test_the_api_host_gets_the_token(monkeypatch) -> None:
235+
"""The check has to still work, or the test below proves nothing."""
236+
request = _captured_request("https://api.github.com/repos/o/r/contents/action.yml", monkeypatch)
237+
assert request.get_header("Authorization") == f"Bearer {SECRET}"
238+
239+
240+
@pytest.mark.parametrize(
241+
"url",
242+
[
243+
# Each of these contains the literal "api.github.com", and none of them
244+
# IS api.github.com. A substring test sent the credential to all three.
245+
"https://elsewhere.example/collect?next=api.github.com",
246+
"https://api.github.com.evil.example/repos/o/r",
247+
"https://evil.example/api.github.com/repos/o/r",
248+
],
249+
)
250+
def test_a_lookalike_host_never_gets_the_token(url: str, monkeypatch) -> None:
251+
request = _captured_request(url, monkeypatch)
252+
assert request.get_header("Authorization") is None, f"the token was sent to {url}"
253+
254+
255+
def test_pypi_does_not_get_a_github_token(monkeypatch) -> None:
256+
"""The same function reaches two hosts; only one of them is authenticated."""
257+
request = _captured_request("https://pypi.org/pypi/tooltrace-bench/json", monkeypatch)
258+
assert request.get_header("Authorization") is None

0 commit comments

Comments
 (0)