Skip to content

Commit a3b74da

Browse files
Copilotfelickz
Copilot
and
felickz
authored
Fix publish summary rendering "❓ no result" for every package (#196)
* Fix publish summary showing "no result" for all packages Root cause: gh api leaks its raw error body (with embedded quotes) to stdout on 404s for brand-new packs, and that raw value was interpolated directly into a jq --arg via ${{ }}, breaking the quoting and corrupting the whole jq invocation into writing a bare boolean instead of a JSON object. That single malformed fragment then aborted jq for every cell. Co-authored-by: felickz <1760475+felickz@users.noreply.github.com> * Remove committed ext/ext-library-sources lockfiles and gitignore them Co-authored-by: felickz <1760475+felickz@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: felickz <1760475+felickz@users.noreply.github.com>
1 parent 3956ba8 commit a3b74da

3 files changed

Lines changed: 67 additions & 28 deletions

File tree

‎.github/scripts/build-publish-summary.sh‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -102,8 +102,13 @@ download_badge() {
102102
cell() {
103103
local lang="$1" type="$2"
104104
local entry
105+
# `type == "object" and` guards against a malformed result fragment (e.g. one job's
106+
# "Record publish result" step writing a bare `true`/`false` instead of a JSON object,
107+
# as happened when publishing v0.7.1 - see PR discussion). Without it, `.language`/`.type`
108+
# would fail to index that non-object element and abort this jq call entirely, which
109+
# previously broke every cell in the table, not just the one for the malformed fragment.
105110
entry=$(echo "$MERGED" | jq -c --arg l "$lang" --arg t "$type" \
106-
'[.[] | select(.language == $l and .type == $t)] | first // empty')
111+
'[.[] | select(type == "object" and .language == $l and .type == $t)] | first // empty')
107112

108113
if [ -z "$entry" ] || [ "$entry" == "null" ]; then
109114
# This combo is expected to publish (unlike the "n/a" combos filtered out

‎.github/workflows/publish.yml‎

Lines changed: 57 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -34,13 +34,19 @@ jobs:
3434
env:
3535
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
3636
run: |
37-
# `|| true` inside the substitution (not after it) matters: `run:` steps default to
38-
# `bash -e`, and a brand-new pack's container doesn't exist on GHCR yet, so `gh api`
39-
# returns 404 (non-zero). Without this, that failure would abort the step here and
40-
# fail the job, blocking the first-ever publish of a new pack. Swallowing it leaves
41-
# PUBLISHED_VERSION empty, which correctly differs from CURRENT_VERSION below and
42-
# triggers a first publish.
43-
PUBLISHED_VERSION=$(gh api /orgs/githubsecuritylab/packages/container/codeql-${{ matrix.language }}-queries/versions --jq '.[0].metadata.container.tags[0]' 2>/dev/null || true)
37+
# A brand-new pack's container doesn't exist on GHCR yet, so `gh api` 404s. On any
38+
# failure, force PUBLISHED_VERSION empty rather than trusting whatever ended up on
39+
# stdout: on error, `gh api` prints the raw (unfiltered) response body there - e.g.
40+
# `{"message":"Package not found.",...}` - even with `--jq` and even with
41+
# `2>/dev/null`. That body's embedded double quotes previously broke out of the
42+
# `--arg previous_version "..."` quoting in the "Record publish result" step below,
43+
# corrupting its jq invocation into writing a bare `true`/`false` instead of a JSON
44+
# object - which then broke the release summary table for every language, not just
45+
# the newly-published one (this is what happened publishing v0.7.1's new go/python
46+
# `ext` packs for the first time).
47+
if ! PUBLISHED_VERSION=$(gh api /orgs/githubsecuritylab/packages/container/codeql-${{ matrix.language }}-queries/versions --jq '.[0].metadata.container.tags[0]' 2>/dev/null); then
48+
PUBLISHED_VERSION=""
49+
fi
4450
CURRENT_VERSION=$(grep -E '^version:' ${{ matrix.language }}/src/qlpack.yml | head -1 | awk '{print $2}')
4551
4652
echo "Published version: $PUBLISHED_VERSION"
@@ -69,6 +75,15 @@ jobs:
6975
7076
- name: Record publish result
7177
if: always()
78+
env:
79+
# Passed via env (not raw `${{ }}` interpolation) since these values ultimately
80+
# derive from external content (a GHCR tag, or `gh api`'s error body on a 404 - see
81+
# the `check_version` step above) that could contain a literal `"`. Interpolating
82+
# such a value directly into `--arg previous_version "${{ ... }}"` below would break
83+
# out of that quoting and corrupt the whole jq invocation; referenced as a shell
84+
# variable instead, it's always passed through safely no matter its content.
85+
PREVIOUS_VERSION: ${{ steps.check_version.outputs.published_version }}
86+
TARGET_VERSION: ${{ steps.check_version.outputs.current_version }}
7287
run: |
7388
mkdir -p publish-results
7489
STATUS="up-to-date"
@@ -85,8 +100,8 @@ jobs:
85100
--arg language "${{ matrix.language }}" \
86101
--arg type "src" \
87102
--arg package "codeql-${{ matrix.language }}-queries" \
88-
--arg previous_version "${{ steps.check_version.outputs.published_version }}" \
89-
--arg target_version "${{ steps.check_version.outputs.current_version }}" \
103+
--arg previous_version "$PREVIOUS_VERSION" \
104+
--arg target_version "$TARGET_VERSION" \
90105
--arg status "$STATUS" \
91106
'{language: $language, type: $type, package: $package, previous_version: $previous_version, target_version: $target_version, status: $status}' \
92107
> "publish-results/${{ matrix.language }}-src.json"
@@ -119,10 +134,11 @@ jobs:
119134
env:
120135
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
121136
run: |
122-
# See the `queries` job above for why `|| true` is inside the substitution: a
123-
# brand-new pack's GHCR container doesn't exist yet, so `gh api` 404s; without this,
124-
# `bash -e` would abort here and block the pack's first-ever publish.
125-
PUBLISHED_VERSION=$(gh api /orgs/githubsecuritylab/packages/container/codeql-${{ matrix.language }}-libs/versions --jq '.[0].metadata.container.tags[0]' 2>/dev/null || true)
137+
# See the `queries` job above for why PUBLISHED_VERSION is forced empty on any
138+
# `gh api` failure instead of relying on `|| true`.
139+
if ! PUBLISHED_VERSION=$(gh api /orgs/githubsecuritylab/packages/container/codeql-${{ matrix.language }}-libs/versions --jq '.[0].metadata.container.tags[0]' 2>/dev/null); then
140+
PUBLISHED_VERSION=""
141+
fi
126142
CURRENT_VERSION=$(grep -E '^version:' ${{ matrix.language }}/lib/qlpack.yml | head -1 | awk '{print $2}')
127143
128144
echo "Published version: $PUBLISHED_VERSION"
@@ -151,6 +167,10 @@ jobs:
151167
152168
- name: Record publish result
153169
if: always()
170+
env:
171+
# See the `queries` job above for why these are passed via env, not `${{ }}`.
172+
PREVIOUS_VERSION: ${{ steps.check_version.outputs.published_version }}
173+
TARGET_VERSION: ${{ steps.check_version.outputs.current_version }}
154174
run: |
155175
mkdir -p publish-results
156176
STATUS="up-to-date"
@@ -167,8 +187,8 @@ jobs:
167187
--arg language "${{ matrix.language }}" \
168188
--arg type "lib" \
169189
--arg package "codeql-${{ matrix.language }}-libs" \
170-
--arg previous_version "${{ steps.check_version.outputs.published_version }}" \
171-
--arg target_version "${{ steps.check_version.outputs.current_version }}" \
190+
--arg previous_version "$PREVIOUS_VERSION" \
191+
--arg target_version "$TARGET_VERSION" \
172192
--arg status "$STATUS" \
173193
'{language: $language, type: $type, package: $package, previous_version: $previous_version, target_version: $target_version, status: $status}' \
174194
> "publish-results/${{ matrix.language }}-lib.json"
@@ -201,10 +221,11 @@ jobs:
201221
env:
202222
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
203223
run: |
204-
# See the `queries` job above for why `|| true` is inside the substitution: a
205-
# brand-new pack's GHCR container doesn't exist yet, so `gh api` 404s; without this,
206-
# `bash -e` would abort here and block the pack's first-ever publish.
207-
PUBLISHED_VERSION=$(gh api /orgs/githubsecuritylab/packages/container/codeql-${{ matrix.language }}-extensions/versions --jq '.[0].metadata.container.tags[0]' 2>/dev/null || true)
224+
# See the `queries` job above for why PUBLISHED_VERSION is forced empty on any
225+
# `gh api` failure instead of relying on `|| true`.
226+
if ! PUBLISHED_VERSION=$(gh api /orgs/githubsecuritylab/packages/container/codeql-${{ matrix.language }}-extensions/versions --jq '.[0].metadata.container.tags[0]' 2>/dev/null); then
227+
PUBLISHED_VERSION=""
228+
fi
208229
CURRENT_VERSION=$(grep -E '^version:' ${{ matrix.language }}/ext/qlpack.yml | head -1 | awk '{print $2}')
209230
210231
echo "Published version: $PUBLISHED_VERSION"
@@ -237,6 +258,10 @@ jobs:
237258
238259
- name: Record publish result
239260
if: always()
261+
env:
262+
# See the `queries` job above for why these are passed via env, not `${{ }}`.
263+
PREVIOUS_VERSION: ${{ steps.check_version.outputs.published_version }}
264+
TARGET_VERSION: ${{ steps.check_version.outputs.current_version }}
240265
run: |
241266
mkdir -p publish-results
242267
STATUS="up-to-date"
@@ -253,8 +278,8 @@ jobs:
253278
--arg language "${{ matrix.language }}" \
254279
--arg type "ext" \
255280
--arg package "codeql-${{ matrix.language }}-extensions" \
256-
--arg previous_version "${{ steps.check_version.outputs.published_version }}" \
257-
--arg target_version "${{ steps.check_version.outputs.current_version }}" \
281+
--arg previous_version "$PREVIOUS_VERSION" \
282+
--arg target_version "$TARGET_VERSION" \
258283
--arg status "$STATUS" \
259284
'{language: $language, type: $type, package: $package, previous_version: $previous_version, target_version: $target_version, status: $status}' \
260285
> "publish-results/${{ matrix.language }}-ext.json"
@@ -288,10 +313,11 @@ jobs:
288313
env:
289314
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
290315
run: |
291-
# See the `queries` job above for why `|| true` is inside the substitution: a
292-
# brand-new pack's GHCR container doesn't exist yet, so `gh api` 404s; without this,
293-
# `bash -e` would abort here and block the pack's first-ever publish.
294-
PUBLISHED_VERSION=$(gh api /orgs/githubsecuritylab/packages/container/codeql-${{ matrix.language }}-library-sources/versions --jq '.[0].metadata.container.tags[0]' 2>/dev/null || true)
316+
# See the `queries` job above for why PUBLISHED_VERSION is forced empty on any
317+
# `gh api` failure instead of relying on `|| true`.
318+
if ! PUBLISHED_VERSION=$(gh api /orgs/githubsecuritylab/packages/container/codeql-${{ matrix.language }}-library-sources/versions --jq '.[0].metadata.container.tags[0]' 2>/dev/null); then
319+
PUBLISHED_VERSION=""
320+
fi
295321
CURRENT_VERSION=$(grep -E '^version:' ${{ matrix.language }}/ext-library-sources/qlpack.yml | head -1 | awk '{print $2}')
296322
297323
echo "Published version: $PUBLISHED_VERSION"
@@ -319,6 +345,10 @@ jobs:
319345
320346
- name: Record publish result
321347
if: always()
348+
env:
349+
# See the `queries` job above for why these are passed via env, not `${{ }}`.
350+
PREVIOUS_VERSION: ${{ steps.check_version.outputs.published_version }}
351+
TARGET_VERSION: ${{ steps.check_version.outputs.current_version }}
322352
run: |
323353
mkdir -p publish-results
324354
STATUS="up-to-date"
@@ -335,8 +365,8 @@ jobs:
335365
--arg language "${{ matrix.language }}" \
336366
--arg type "ext-library-sources" \
337367
--arg package "codeql-${{ matrix.language }}-library-sources" \
338-
--arg previous_version "${{ steps.check_version.outputs.published_version }}" \
339-
--arg target_version "${{ steps.check_version.outputs.current_version }}" \
368+
--arg previous_version "$PREVIOUS_VERSION" \
369+
--arg target_version "$TARGET_VERSION" \
340370
--arg status "$STATUS" \
341371
'{language: $language, type: $type, package: $package, previous_version: $previous_version, target_version: $target_version, status: $status}' \
342372
> "publish-results/${{ matrix.language }}-ext-library-sources.json"

‎.gitignore‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,10 @@
99
*.actual
1010
*.class
1111
*/*.class
12+
# Lockfiles for model/extension packs (ext, ext-library-sources) should never
13+
# be generated or committed; see CONTRIBUTING.md's "Releases & publishing" section.
14+
**/ext/codeql-pack.lock.yml
15+
**/ext-library-sources/codeql-pack.lock.yml
1216
# Cloned repository of codeql
1317
/codeql
1418
# Downloaded/extracted CodeQL CLI (see .github/actions/install-codeql), cached

0 commit comments

Comments
 (0)