Skip to content

Commit c349de9

Browse files
felickzCopilot
andcommitted
fix: use open-ended extensionTargets floors instead of exact pins (#206)
Root cause of #206: extensionTargets is exact-pinned to whatever codeql/<lang>-all version this repo's CI happens to test against, and re-pinned on every CLI bump. Unlike a normal dependencies: mismatch (which fails loudly at install time), an unsatisfied extensionTargets constraint makes the CLI silently drop the entire extension pack - zero data-extension rows applied, only a low-visibility WARNING: ... is unused, job stays green. Any drift between this repo's tested CLI version and a consumer's actual CLI version turns into silent, undetected loss of coverage. Confirmed via local CLI testing, a genuine GHCR registry install, and a real hosted GitHub Actions default-setup run. By contrast, a genuinely unsatisfiable dependencies: constraint fails hard (ERROR: No valid pack solution found...), and even a stale/incompatible dependencies pin on the CLI-bundled codeql/<lang>-all itself is harmless - the CLI always resolves that dependency to whatever -all it actually bundles regardless of the declared range, and any real incompatibility then surfaces as an ordinary QL compile error. So extensionTargets is the only mechanism with this silent failure mode - confirmed empirically with isolated dependency-resolution tests. Changes (piloting on Java first): - java/ext and java/ext-library-sources: extensionTargets changed from an exact pin (9.2.2) to an open-ended floor (>=9.2.2, no upper bound). Verified this resolves correctly against CLI 2.26.2's bundled java-all 9.2.2, and continues to work as the CLI/bundle moves forward. - pin-codeql-library-versions.sh: excludes */ext and */ext-library-sources from its auto-repin loop entirely, so a routine CLI bump can never silently re-tighten an extensionTargets floor back to an exact version. Query/library pack dependencies: pinning is unchanged - that serves a different purpose (preventing codeql pack upgrade from resolving unconstrained '*' deps to registry-latest). - update-codeql-version.yml: updated comments/PR body text to reflect the narrower scope of the auto-repin step. - CONTRIBUTING.md: documents the new extensionTargets floor policy in a new warning box, cross-referenced from the existing ext-packs-no-install note. Other languages (csharp, go, python) still use an exact extensionTargets pin pending the same conversion in follow-up work. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: f02ef23b-e35d-4536-bd58-f2fbc2976ba9
1 parent a68fd6d commit c349de9

5 files changed

Lines changed: 110 additions & 22 deletions

File tree

‎.github/scripts/pin-codeql-library-versions.sh‎

Lines changed: 49 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,38 @@
11
#!/usr/bin/env bash
2-
# Pins every `codeql/<name>` dependency in this repo's qlpack.yml files to the
3-
# exact library version shipped in the official CodeQL Bundle for a given CLI
2+
# Pins every `codeql/<name>` `dependencies:` entry in this repo's query/library
3+
# pack qlpack.yml files (`<language>/src`, `<language>/lib`, etc.) to the exact
4+
# library version shipped in the official CodeQL Bundle for a given CLI
45
# release - overwriting whatever value is currently there (an unconstrained
56
# `'*'`, or an exact version pinned by a previous run of this script against
67
# an older CLI).
78
#
8-
# Why this exists: `codeql pack upgrade` resolves an unconstrained `'*'`
9-
# dependency to the *latest-ever-published* version in the configured
10-
# registry (GHCR) - completely independent of whatever CodeQL CLI version is
11-
# pinned in `.codeqlversion`. That mismatch can jump `codeql/<lang>-all`
12-
# several major versions ahead of what the pinned CLI actually ships/tests
13-
# against, which can silently break analyses (see CONTRIBUTING.md's
14-
# "Updating the pinned CodeQL CLI/library version" section for a worked
15-
# example). Pinning these `codeql/*` deps to the exact bundle-paired version
16-
# before running `codeql pack upgrade` makes that resolution deterministic
17-
# and keeps every pack's declared library dependency in lockstep with the
18-
# CLI version this repo says it supports.
9+
# Deliberately does NOT touch `<language>/ext` or `<language>/ext-library-sources`
10+
# - those are model/extension packs that declare `extensionTargets:`, not
11+
# `dependencies:`, and are excluded from this script's file list entirely. See
12+
# CONTRIBUTING.md's "Supported CodeQL versions" section ("Why `extensionTargets`
13+
# is a floor, not an exact pin") for why: unlike `dependencies:`, an unsatisfied
14+
# `extensionTargets` constraint doesn't fail loudly, it silently drops the whole
15+
# extension pack, so re-pinning it to an exact version on every CLI bump (the
16+
# way this script handles `dependencies:`) turns routine consumer CLI drift into
17+
# silent, undetected loss of coverage (GitHubSecurityLab/CodeQL-Community-Packs#206).
18+
# `extensionTargets` floors are maintained by hand instead, as open-ended ranges.
19+
#
20+
# Why this exists (for `dependencies:`): `codeql pack upgrade` resolves an
21+
# unconstrained `'*'` dependency to the *latest-ever-published* version in the
22+
# configured registry (GHCR) - completely independent of whatever CodeQL CLI
23+
# version is pinned in `.codeqlversion`. That mismatch can jump
24+
# `codeql/<lang>-all` several major versions ahead of what the pinned CLI
25+
# actually ships/tests against, which can silently break analyses (see
26+
# CONTRIBUTING.md's "Updating the pinned CodeQL CLI/library version" section
27+
# for a worked example). Pinning these `codeql/*` deps to the exact
28+
# bundle-paired version before running `codeql pack upgrade` makes that
29+
# resolution deterministic and keeps every pack's declared library dependency
30+
# in lockstep with the CLI version this repo says it supports. This is safe
31+
# for `dependencies:` (unlike `extensionTargets:`) because a genuinely
32+
# unsatisfiable `dependencies:` constraint fails loudly at `codeql pack
33+
# install`/`resolve-dependencies` time (`ERROR: No valid pack solution
34+
# found...`), and any incompatibility in the CLI-bundled `codeql/<lang>-all`
35+
# itself surfaces as an ordinary QL compile error - never a silent drop.
1936
#
2037
# This script is idempotent and must be re-run (with a new target version)
2138
# on every subsequent CLI bump: once a dependency is pinned to an exact
@@ -67,8 +84,25 @@ echo "Discovered $(wc -l < "$VERSIONS_FILE") bundled codeql/* packages for CLI $
6784
# artifacts (.codeql/ pack caches, the /codeql cloned-repo checkout dir, and
6885
# /codeql_home, where .github/actions/install-codeql downloads/extracts the
6986
# CodeQL CLI - which ships its own small vendored qlpack.yml packs, e.g.
70-
# codeql/<lang>/downgrades, that have nothing to do with this repo).
71-
mapfile -t QLPACK_FILES < <(find . -name qlpack.yml -not -path "*/.codeql/*" -not -path "./codeql/*" -not -path "./codeql_home/*")
87+
# codeql/<lang>/downgrades, that have nothing to do with this repo) - and
88+
# excluding every `<language>/ext` and `<language>/ext-library-sources`
89+
# model/extension pack. Those use `extensionTargets:` instead of
90+
# `dependencies:`, and deliberately do NOT get re-pinned to the exact bundle
91+
# version on every CLI bump the way `dependencies:` does here (see
92+
# CONTRIBUTING.md's "Supported CodeQL versions" section, "Why
93+
# `extensionTargets` is a floor, not an exact pin"). Unlike `dependencies:`,
94+
# an unsatisfied `extensionTargets` constraint doesn't fail loudly - the CLI
95+
# just silently drops the whole extension pack (zero data-extension rows
96+
# applied, only a low-visibility `WARNING: ... is unused`) - so constantly
97+
# re-pinning it to whatever this repo's own CI happens to test against turns
98+
# every routine consumer CLI/version mismatch into silent, undetected loss of
99+
# coverage (see GitHubSecurityLab/CodeQL-Community-Packs#206). Instead,
100+
# `extensionTargets` is set by hand as an open-ended floor
101+
# (`codeql/<lang>-all: '>=X.Y.Z'`) and only ever raised when a maintainer
102+
# confirms an actual breaking change to the models-as-data schema (e.g. an
103+
# extensible predicate's arity/column set changed) - our own test suite
104+
# failing is the trigger to look for that, not a routine CLI bump.
105+
mapfile -t QLPACK_FILES < <(find . -name qlpack.yml -not -path "*/.codeql/*" -not -path "./codeql/*" -not -path "./codeql_home/*" -not -path "*/ext/*" -not -path "*/ext-library-sources/*")
72106

73107
declare -A PINNED_COUNT=()
74108
while read -r pkg ver; do

‎.github/workflows/update-codeql-version.yml‎

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,12 @@ jobs:
8484
env:
8585
GH_TOKEN: ${{ github.token }}
8686
run: |
87+
# NOTE: this only re-pins `dependencies:` in query/library packs
88+
# (<language>/src, <language>/lib, etc.) - it deliberately skips
89+
# <language>/ext and <language>/ext-library-sources, whose
90+
# `extensionTargets:` floors are maintained by hand (see
91+
# pin-codeql-library-versions.sh's header comment and
92+
# CONTRIBUTING.md's "Supported CodeQL versions" section).
8793
.github/scripts/pin-codeql-library-versions.sh "${{ steps.version.outputs.version }}"
8894
8995
- name: Upgrade every pack's dependencies
@@ -234,11 +240,13 @@ jobs:
234240
echo
235241
echo "This PR:"
236242
echo '- Updates `.codeqlversion` to `'"${{ steps.version.outputs.version }}"'`.'
237-
echo '- Pins every `codeql/<lang>-all` / `codeql/<lang>-queries` dependency across all'
238-
echo ' `qlpack.yml` files to the exact version shipped in the official CodeQL Bundle'
239-
echo ' for this CLI release (see `.github/scripts/pin-codeql-library-versions.sh`) -'
240-
echo ' this keeps `codeql pack upgrade` from jumping those libraries to'
241-
echo ' registry-latest instead of the version this CLI actually ships/tests against.'
243+
echo '- Pins every `codeql/<lang>-all` / `codeql/<lang>-queries` `dependencies:`'
244+
echo ' entry across query/library `qlpack.yml` files (not `*/ext` or'
245+
echo ' `*/ext-library-sources`, whose `extensionTargets` floors are maintained by'
246+
echo ' hand) to the exact version shipped in the official CodeQL Bundle for this CLI'
247+
echo ' release (see `.github/scripts/pin-codeql-library-versions.sh`) - this keeps'
248+
echo ' `codeql pack upgrade` from jumping those libraries to registry-latest instead'
249+
echo ' of the version this CLI actually ships/tests against.'
242250
echo '- Runs `codeql pack upgrade <dir>` for every pack directory to refresh its'
243251
echo ' `codeql-pack.lock.yml` against the new CLI and pinned library versions.'
244252
echo

‎CONTRIBUTING.md‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -118,6 +118,47 @@ The pinning is codified per language across:
118118
> this shortly after this pinning behavior was introduced and a pack's `qlpack.yml` still shows
119119
> `codeql/<pkg>: '*'`, that just means its dependencies haven't been re-resolved since, not that the
120120
> convention doesn't apply to it.
121+
>
122+
> **This exact-pin treatment applies to `dependencies:` only** (query/library packs: `src`, `lib`,
123+
> etc.) - `<language>/ext` and `<language>/ext-library-sources` model/extension packs use
124+
> `extensionTargets:` instead, and are handled completely differently; see the next warning box.
125+
126+
> [!WARNING]
127+
> <a id="extensiontargets-floor"></a>**Why `extensionTargets` is an open-ended floor, not an exact
128+
> pin (unlike `dependencies:` above):** `extensionTargets` and `dependencies:` look similar but fail
129+
> very differently when a consumer's CodeQL CLI doesn't bundle the exact version this repo pinned
130+
> against. An unsatisfiable `dependencies:` constraint fails **loudly** - `codeql pack
131+
> install`/`resolve-dependencies` refuses with `ERROR: No valid pack solution found...` - and even a
132+
> stale/incompatible pin on the CLI-bundled `codeql/<lang>-all` itself is harmless in practice,
133+
> because the CLI always resolves that specific dependency to whatever `-all` version it actually
134+
> bundles regardless of the declared range, and any genuine incompatibility then surfaces as an
135+
> ordinary QL compile error. An unsatisfiable `extensionTargets` constraint does neither: the CLI
136+
> just **silently drops the entire extension pack** - zero data-extension rows applied, no compile
137+
> error, no SARIF-level signal, job stays green - and surfaces only a low-visibility
138+
> `WARNING: Extension pack '<name>' is unused.` that's easy to miss in CI logs. Exact-pinning
139+
> `extensionTargets` to whatever this repo's own CI happens to test against on every routine CLI bump
140+
> (the way [`pin-codeql-library-versions.sh`][pin-codeql-library-versions-script] handles
141+
> `dependencies:`) therefore turns *any* drift between this repo's tested CLI version and a
142+
> consumer's actual CLI version into silent, undetected loss of coverage - this is exactly what
143+
> happened in [#206][issue-206], reproduced and confirmed there across a local CLI, a genuine GHCR
144+
> registry install, and a real hosted GitHub Actions run.
145+
>
146+
> The fix: `extensionTargets` is set **by hand** as an open-ended floor
147+
> (`codeql/<lang>-all: '>=X.Y.Z'`, no upper bound) instead of an exact pin, and
148+
> [`pin-codeql-library-versions.sh`][pin-codeql-library-versions-script] explicitly excludes
149+
> `*/ext` and `*/ext-library-sources` from its rewrite loop so a routine CLI bump can never silently
150+
> re-tighten it back to an exact version. An open floor works because the models-as-data extensible
151+
> predicates (`sinkModel`, `sourceModel`, `summaryModel`, etc.) these packs contribute rows to are a
152+
> stable schema that only changes on a genuine breaking change (an extensible predicate's
153+
> arity/column set changing) - which is rare and would show up as real compilation/test failures in
154+
> this repo's own CI, not something that needs to be guarded against on every release. Only raise an
155+
> `extensionTargets` floor by hand when there's evidence of an actual such breaking change (upstream
156+
> `github/codeql` CHANGELOG "Breaking Changes" entries are a good place to check, though in practice
157+
> most of those are QL library API changes that affect query/library packs, not the data-extension
158+
> row format) - not just because a newer CLI happens to be available. As of this writing, Java's
159+
> `<language>/ext`/`ext-library-sources` packs pilot this open-floor approach; other languages still
160+
> use an exact `extensionTargets` pin pending the same conversion.
161+
121162

122163
**This section no longer hand-maintains a version table** - it used to, but that table went stale
123164
across multiple CLI bumps in a row (nobody remembered to update it, and there was no CI check
@@ -313,6 +354,10 @@ coding agent) in the loop for the hard part — fixing whatever the new CLI brea
313354
> If you run `codeql pack install`/`upgrade` against one of these directories locally while
314355
> developing (e.g. to sanity-check a new data extension), delete the resulting
315356
> `codeql-pack.lock.yml` before committing.
357+
>
358+
> This is a separate concern from how `extensionTargets`' *version value* itself is chosen and
359+
> maintained — see the ["Why `extensionTargets` is an open-ended floor, not an exact
360+
> pin"](#extensiontargets-floor) warning box above.
316361
317362
> [!WARNING]
318363
> The `.codeqlversion` bump and the pack version bumps don't have to land in the same PR, but
@@ -463,3 +508,4 @@ Please do get in touch (privacy@github.com) if you have any questions about this
463508
[pr-155]: https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/pull/155
464509
[pr-158]: https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/pull/158
465510
[pr-159]: https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/pull/159
511+
[issue-206]: https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/issues/206

‎java/ext-library-sources/qlpack.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ library: true
22
name: githubsecuritylab/codeql-java-library-sources
33
version: 0.7.3
44
extensionTargets:
5-
codeql/java-all: '9.2.2'
5+
codeql/java-all: '>=9.2.2'
66
dataExtensions:
77
- 'manual/*.yml'
88
- 'manual/**/*.yml'

‎java/ext/qlpack.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ library: true
22
name: githubsecuritylab/codeql-java-extensions
33
version: 0.7.3
44
extensionTargets:
5-
codeql/java-all: '9.2.2'
5+
codeql/java-all: '>=9.2.2'
66
dataExtensions:
77
- 'manual/*.yml'
88
- 'manual/**/*.yml'

0 commit comments

Comments
 (0)