Skip to content

Commit 5b6f848

Browse files
Copilotfelickz
andauthored
Fix is not an extension target of warning for extension packs (#191)
* Fix 'is not an extension target of' warning for extension packs Co-authored-by: felickz <1760475+felickz@users.noreply.github.com> * Investigate python extension pack failure Co-authored-by: felickz <1760475+felickz@users.noreply.github.com> * Restore extension pack lockfile deletions Co-authored-by: felickz <1760475+felickz@users.noreply.github.com> * placeholder - starting python model schema migration Co-authored-by: felickz <1760475+felickz@users.noreply.github.com> * Fix Python model schema: merge 4-column source/sinkModel to 3-column for codeql/python-all 7.2.x, drop malformed summaryModel row; clarify update-codeql-version.yml comment Co-authored-by: felickz <1760475+felickz@users.noreply.github.com> * Remove ext/ext-library-sources lock files re-added by placeholder commit Co-authored-by: felickz <1760475+felickz@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: felickz <1760475+felickz@users.noreply.github.com>
1 parent 55128cf commit 5b6f848

17 files changed

Lines changed: 2469 additions & 1289 deletions

‎.github/scripts/pr-suites-packs.sh‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,16 @@ if [[ -z "$PR_NUMBER" ]]; then
2222
for pack_dir in "$LANGUAGE/src" "$LANGUAGE/lib" "$LANGUAGE/ext" "$LANGUAGE/ext-library-sources"; do
2323
if [[ -f "$pack_dir/qlpack.yml" ]]; then
2424
echo "[+] Compiling Pack: $pack_dir"
25-
codeql pack install "$pack_dir"
25+
# `ext`/`ext-library-sources` are CodeQL model/extension packs (`extensionTargets`,
26+
# no `dependencies`) - deliberately skip `codeql pack install` for them. Installing
27+
# writes a `codeql-pack.lock.yml` with an empty `dependencies: {}` map, and a
28+
# checked-in lock file in that state makes `codeql pack create` emit a bogus
29+
# `addsTo.pack '...' is not an extension target of '...'` warning for every data
30+
# extension in the pack (a known CodeQL CLI bug, see
31+
# https://github.com/github/codeql/issues/20211). See CONTRIBUTING.md.
32+
if [[ "$pack_dir" != "$LANGUAGE/ext" && "$pack_dir" != "$LANGUAGE/ext-library-sources" ]]; then
33+
codeql pack install "$pack_dir"
34+
fi
2635
codeql pack create "$pack_dir"
2736
fi
2837
done

‎.github/workflows/ci.yml‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -189,7 +189,13 @@ jobs:
189189
GITHUB_TOKEN: ${{ github.token }}
190190
run: |
191191
gh extension install github/gh-codeql
192-
gh codeql pack install "${{ matrix.language }}/ext/"
192+
# NOTE: deliberately no `gh codeql pack install` here. `ext` is a CodeQL model/extension
193+
# pack (`library: true`, `extensionTargets`, no `dependencies`) - installing it just
194+
# (re)writes a `codeql-pack.lock.yml` with an empty `dependencies: {}` map, and a
195+
# checked-in lock file in that state makes a subsequent `codeql pack create` emit a
196+
# bogus `addsTo.pack '...' is not an extension target of '...'` warning for every data
197+
# extension in the pack (a known CodeQL CLI bug, see
198+
# https://github.com/github/codeql/issues/20211). See CONTRIBUTING.md.
193199
gh codeql pack create "${{ matrix.language }}/ext/"
194200
195201
library-sources:
@@ -221,7 +227,8 @@ jobs:
221227
env:
222228
GITHUB_TOKEN: ${{ github.token }}
223229
run: |
224-
codeql pack install "${{ matrix.language }}/ext-library-sources/"
230+
# NOTE: deliberately no `codeql pack install` here - see the matching comment in the
231+
# `extensions` job above.
225232
codeql pack create "${{ matrix.language }}/ext-library-sources/"
226233
227234
configs:

‎.github/workflows/publish.yml‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -226,7 +226,13 @@ jobs:
226226
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
227227
run: |
228228
echo "Publishing codeql-${{ matrix.language }}-extensions."
229-
codeql pack install "${{ matrix.language }}/ext"
229+
# NOTE: deliberately no `codeql pack install` here. `ext` is a CodeQL model/extension
230+
# pack (`library: true`, `extensionTargets`, no `dependencies`) - installing it just
231+
# (re)writes a `codeql-pack.lock.yml` with an empty `dependencies: {}` map, and a
232+
# checked-in lock file in that state makes a subsequent `codeql pack publish`/`create`
233+
# emit a bogus `addsTo.pack '...' is not an extension target of '...'` warning for every
234+
# data extension in the pack (a known CodeQL CLI bug, see
235+
# https://github.com/github/codeql/issues/20211). See CONTRIBUTING.md.
230236
codeql pack publish "${{ matrix.language }}/ext"
231237
232238
- name: Record publish result
@@ -307,7 +313,8 @@ jobs:
307313
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
308314
run: |
309315
echo "Publishing codeql-${{ matrix.language }}-library-sources."
310-
codeql pack install "${{ matrix.language }}/ext-library-sources"
316+
# NOTE: deliberately no `codeql pack install` here - see the matching comment in the
317+
# `extensions` job above.
311318
codeql pack publish "${{ matrix.language }}/ext-library-sources"
312319
313320
- name: Record publish result

‎.github/workflows/update-codeql-version.yml‎

Lines changed: 14 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -91,8 +91,10 @@ jobs:
9191
GITHUB_TOKEN: ${{ github.token }}
9292
run: |
9393
set -euo pipefail
94-
# Exclusions (kept in sync with the same exclusions in
95-
# .github/scripts/pin-codeql-library-versions.sh):
94+
# Exclusions (the first three below are also excluded in
95+
# .github/scripts/pin-codeql-library-versions.sh; the last one is
96+
# NOT - ext/ext-library-sources still need their extensionTargets
97+
# versions pinned, but must be skipped here to avoid lock-file churn):
9698
# - ./ql/hotspots is a standalone local dev tool (see
9799
# ql/hotspots/README.md and .github/workflows/hotspots.yml), not
98100
# one of the per-language src/lib/ext/ext-library-sources packs
@@ -106,7 +108,16 @@ jobs:
106108
# - ./codeql and */.codeql are a locally-cloned github/codeql
107109
# checkout (used by ql/hotspots) and CodeQL's own per-pack build
108110
# caches, respectively - neither is a repo pack either.
109-
for dir in $(find . -name qlpack.yml -not -path "./ql/hotspots/*" -not -path "./codeql_home/*" -not -path "./codeql/*" -not -path "*/.codeql/*" -exec dirname {} \;); do
111+
# - <lang>/ext and <lang>/ext-library-sources are CodeQL model/extension packs
112+
# (`extensionTargets`, no `dependencies`) - NOT excluded from
113+
# pin-codeql-library-versions.sh (their extensionTargets versions still need
114+
# pinning), but excluded here: `codeql pack upgrade` (re)writes their
115+
# codeql-pack.lock.yml with an empty `dependencies: {}` map, and a checked-in
116+
# lock file in that state makes a later `codeql pack create`/`publish` emit a
117+
# bogus `addsTo.pack '...' is not an extension target of '...'` warning for
118+
# every data extension in the pack (a known CodeQL CLI bug, see
119+
# https://github.com/github/codeql/issues/20211). See CONTRIBUTING.md.
120+
for dir in $(find . -name qlpack.yml -not -path "./ql/hotspots/*" -not -path "./codeql_home/*" -not -path "./codeql/*" -not -path "*/.codeql/*" -not -path "*/ext/*" -not -path "*/ext-library-sources/*" -exec dirname {} \;); do
110121
echo "::group::codeql pack upgrade $dir"
111122
codeql pack upgrade "$dir"
112123
echo "::endgroup::"

‎CONTRIBUTING.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -293,6 +293,27 @@ coding agent) in the loop for the hard part — fixing whatever the new CLI brea
293293
> `codeql pack upgrade`/the pinning script can never resolve it. This is a pre-existing, unrelated
294294
> quirk of that tool, not something the version-bump automation needs to (or can) fix.
295295
296+
> [!NOTE]
297+
> <a id="ext-packs-no-install"></a>**Why `<language>/ext` and `<language>/ext-library-sources` are
298+
> never `codeql pack install`ed or `codeql pack upgrade`d:** these are CodeQL
299+
> [model/extension packs](https://docs.github.com/en/code-security/tutorials/customize-code-scanning/create-and-work-with-codeql-packs#creating-a-codeql-model-pack)
300+
> — `library: true`, an `extensionTargets` map, and (by design) **no `dependencies`**. `codeql pack
301+
> install`/`codeql pack upgrade` still (re)write a `codeql-pack.lock.yml` for them, but since there's
302+
> nothing to resolve it's always an empty `dependencies: {}` map. A *checked-in* lock file in that
303+
> state triggers a known CodeQL CLI bug
304+
> ([github/codeql#20211](https://github.com/github/codeql/issues/20211)): a later `codeql pack
305+
> create`/`codeql pack publish` on the same pack emits a bogus `WARNING: In extension for
306+
> codeql/<language>-all:<extensible>, addsTo.pack 'codeql/<language>-all' is not an extension target
307+
> of '...'` for every data extension file in the pack, even though `extensionTargets` correctly lists
308+
> that pack. CI (`ci.yml`'s `extensions`/`library-sources` jobs, `publish.yml`'s `extensions`/
309+
> `library_sources_extensions` jobs, `pr-suites-packs.sh`) and
310+
> [`update-codeql-version.yml`](#updating-the-pinned-codeql-clilibrary-version)'s `codeql pack
311+
> upgrade` loop deliberately skip `install`/`upgrade` for these two pack types and go straight to
312+
> `codeql pack create`/`publish` — and no `codeql-pack.lock.yml` should ever be committed for them.
313+
> If you run `codeql pack install`/`upgrade` against one of these directories locally while
314+
> developing (e.g. to sanity-check a new data extension), delete the resulting
315+
> `codeql-pack.lock.yml` before committing.
316+
296317
> [!WARNING]
297318
> The `.codeqlversion` bump and the pack version bumps don't have to land in the same PR, but
298319
> splitting them is risky: [#124][pr-124] refreshed `.codeqlversion` and every language's

‎csharp/ext-library-sources/codeql-pack.lock.yml‎

Lines changed: 0 additions & 4 deletions
This file was deleted.

‎csharp/ext/codeql-pack.lock.yml‎

Lines changed: 0 additions & 4 deletions
This file was deleted.

‎go/ext/codeql-pack.lock.yml‎

Lines changed: 0 additions & 4 deletions
This file was deleted.

‎java/ext-library-sources/codeql-pack.lock.yml‎

Lines changed: 0 additions & 4 deletions
This file was deleted.

‎java/ext/codeql-pack.lock.yml‎

Lines changed: 0 additions & 4 deletions
This file was deleted.

0 commit comments

Comments
 (0)