Skip to content

Commit 61bb4ba

Browse files
authored
ci(release): run the release status and vendor-update reads from util/ scripts (#1967) (#2070)
Step 2 of #1967, second batch for create-release.yml: the last two inline shell steps move into stdlib-only scripts, with each step's env:, if:, needs:, outputs and timeout-minutes unchanged. * util/release_status.py: "Report why this run did or did not release anything" in release_status. * util/release_vendor_update.py: "Read whether its vendor-update job ran" in vendor_update. * Both jobs gain contents: read and a checkout of util/ alone at github.sha, without credentials, ahead of the script step. * workflows.rst lists both scripts. tests/project/test_release_status_scripts_1967.py runs each former shell, kept verbatim, and its script on the same cases, and requires the same status, output, $GITHUB_OUTPUT, step summary and gh calls; the case tables reach every branch of each shell. The tests that ran or pinned the shell now run or read the scripts, and find the steps by name.
1 parent 27288aa commit 61bb4ba

12 files changed

Lines changed: 1109 additions & 205 deletions

‎.github/workflows/create-release.yml‎

Lines changed: 27 additions & 157 deletions
Original file line numberDiff line numberDiff line change
@@ -136,41 +136,34 @@ jobs:
136136
# in that run's place, so the displaced run's release waits for the next run
137137
# on ``main``: the tip's own Vendor Update if the tip's Unit Tests run has not
138138
# completed yet, else the next push's or Saturday's.
139+
#
140+
# #1967: util/release_vendor_update.py holds the step's shell, so the job
141+
# checks out util/ alone, for it and util/_actions.py, which is all
142+
# ``contents: read`` is for.
139143
vendor_update:
140144
name: Triggering Vendor Update
141145
runs-on: ubuntu-latest
142146
if: ${{ github.event_name == 'workflow_run' && github.event.workflow_run.conclusion == 'success' }}
143147
timeout-minutes: 5
144148
permissions:
145149
actions: read
150+
contents: read
146151
outputs:
147152
updated: ${{ steps.updated.outputs.updated }}
148153
steps:
154+
- name: Checkout code
155+
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
156+
with:
157+
ref: ${{ github.sha }}
158+
sparse-checkout: util
159+
persist-credentials: false
160+
149161
- name: Read whether its vendor-update job ran
150162
id: updated
151163
env:
152164
GH_TOKEN: ${{ github.token }}
153165
PCAPKIT_VENDOR_RUN_ID: ${{ github.event.workflow_run.id }}
154-
run: |
155-
set -uo pipefail
156-
157-
if ! result="$(gh api "repos/${GITHUB_REPOSITORY}/actions/runs/${PCAPKIT_VENDOR_RUN_ID}/jobs" \
158-
--paginate --jq '.jobs[] | select(.name == "vendor-update") | .conclusion')"; then
159-
echo "::warning title=Vendor Update not read (#1748)::could not list the jobs of Vendor Update run ${PCAPKIT_VENDOR_RUN_ID}, so this run treats it as one that updated."
160-
exit 0
161-
fi
162-
case "$result" in
163-
success)
164-
echo "updated=true" >> "$GITHUB_OUTPUT"
165-
;;
166-
skipped)
167-
echo "updated=false" >> "$GITHUB_OUTPUT"
168-
echo "Vendor Update run ${PCAPKIT_VENDOR_RUN_ID} skipped vendor-update: its verdict did not cover the commit it checked out."
169-
;;
170-
*)
171-
echo "::warning title=Vendor Update not read (#1748)::Vendor Update run ${PCAPKIT_VENDOR_RUN_ID} reports vendor-update as '${result//$'\n'/ }', so this run treats it as one that updated."
172-
;;
173-
esac
166+
run: python3 util/release_vendor_update.py
174167

175168
version_check:
176169
name: Check Version
@@ -1199,15 +1192,25 @@ jobs:
11991192
# chosen over ``always()`` so that cancelling the run cancels this report too
12001193
# rather than leaving it to describe a run it cannot fully see.
12011194
#
1202-
# No ``environment:`` and ``permissions: {}``: it only annotates, from the
1203-
# ``needs:`` context and the trigger event.
1195+
# No ``environment:``: it only annotates, from the ``needs:`` context and the
1196+
# trigger event. #1967: util/release_status.py holds the step's shell, so the
1197+
# job checks out util/ alone, for it and util/_actions.py, which is all
1198+
# ``contents: read`` is for.
12041199
release_status:
12051200
name: Release status
12061201
runs-on: ubuntu-latest
12071202
needs: [ vendor_update, unit-tests, version_check, github, tag, pypi, conda ]
12081203
if: ${{ !cancelled() }}
1209-
permissions: {}
1204+
permissions:
1205+
contents: read
12101206
steps:
1207+
- name: Checkout code
1208+
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
1209+
with:
1210+
ref: ${{ github.sha }}
1211+
sparse-checkout: util
1212+
persist-credentials: false
1213+
12111214
- name: Report why this run did or did not release anything
12121215
env:
12131216
PCAPKIT_EVENT_NAME: ${{ github.event_name }}
@@ -1231,137 +1234,4 @@ jobs:
12311234
PCAPKIT_RELEASED_ELSEWHERE: ${{ needs.version_check.outputs.PCAPKIT_RELEASED_ELSEWHERE }}
12321235
PCAPKIT_PYPI_KTLO_310: ${{ needs.pypi.outputs.ktlo_310 }}
12331236
PCAPKIT_PYPI_KTLO_PYPY310: ${{ needs.pypi.outputs.ktlo_pypy310 }}
1234-
run: |
1235-
set -euo pipefail
1236-
1237-
# #1633: pypi's KTLO legs reach neither needs.pypi.result
1238-
# (continue-on-error) nor PCAPKIT_PYPI_COMPLETE's count, so each is
1239-
# only reported here, as a notice, from its own job output.
1240-
ktlo_310="${PCAPKIT_PYPI_KTLO_310:-}"
1241-
ktlo_pypy310="${PCAPKIT_PYPI_KTLO_PYPY310:-}"
1242-
if [ -n "${ktlo_310}${ktlo_pypy310}" ]; then
1243-
echo "::notice title=KTLO legs (non-blocking)::pypi 3.10=${ktlo_310:-not run} pypy3.10=${ktlo_pypy310:-not run}. End-of-life Python, built and published but never blocking (#1633); neither result changes this run's verdict."
1244-
fi
1245-
1246-
# Case 1: version_check never ran. On `workflow_run`, Vendor Update did
1247-
# not conclude `success`: `skipped`, nothing to update, gets a quiet
1248-
# notice; anything else a warning, as Vendor Update needs attention --
1249-
# but no tag exists to strand, so it is not #888. On a push it is quiet
1250-
# only for the release App's own tag (#1634); an unset bot id never
1251-
# matches. #1748: a `success` that never ran vendor-update is quiet too.
1252-
if [ "$PCAPKIT_VERSION_CHECK" = "skipped" ]; then
1253-
if [ "$PCAPKIT_EVENT_NAME" = "push" ] && [ -n "$PCAPKIT_RELEASE_APP_BOT_ID" ] \
1254-
&& [ "$PCAPKIT_ACTOR_ID" = "$PCAPKIT_RELEASE_APP_BOT_ID" ]; then
1255-
echo "::notice title=Nothing to release::the release App's bot pushed this tag from the Create Release run that released its commit, so this run correctly released nothing (#1634)."
1256-
elif [ "$PCAPKIT_EVENT_NAME" = "workflow_run" ] && [ "$PCAPKIT_WORKFLOW_RUN_CONCLUSION" = "skipped" ]; then
1257-
echo "::notice title=Nothing to release::the triggering Vendor Update run found nothing to update, so this run correctly released nothing."
1258-
elif [ "$PCAPKIT_EVENT_NAME" = "workflow_run" ] && [ "$PCAPKIT_WORKFLOW_RUN_CONCLUSION" = "success" ] \
1259-
&& [ "${PCAPKIT_VENDOR_UPDATED:-}" = "false" ]; then
1260-
echo "::notice title=Nothing to release::the triggering Vendor Update run skipped vendor-update, since its Unit Tests verdict did not cover the commit it checked out, so it updated nothing and this run correctly released nothing (#1748)."
1261-
elif [ "$PCAPKIT_EVENT_NAME" = "workflow_run" ]; then
1262-
echo "::warning title=Release skipped::the triggering Vendor Update run concluded '${PCAPKIT_WORKFLOW_RUN_CONCLUSION}', not 'success', so version_check, the release test gate and every release job were skipped. Not #888 -- no tag was created that could be left stranded -- but Vendor Update likely needs attention."
1263-
else
1264-
echo "::warning title=Release skipped for an unexpected reason::version_check reported 'skipped' with event_name='${PCAPKIT_EVENT_NAME}', which is not the workflow_run non-success case this job otherwise expects. Check the version_check job's own if: condition."
1265-
fi
1266-
exit 0
1267-
fi
1268-
1269-
# version_check ran and did not succeed: a workflow failure, not #888.
1270-
if [ "$PCAPKIT_VERSION_CHECK" != "success" ]; then
1271-
echo "::warning title=Release blocked before it could start::version_check did not succeed (result: '${PCAPKIT_VERSION_CHECK}'), so neither the release test gate nor any release job ran. This is a workflow failure, not #888."
1272-
exit 0
1273-
fi
1274-
1275-
# The gate did not pass, so every release job skipped: a test failure,
1276-
# not #888, and already red in this run's checks.
1277-
if [ "$PCAPKIT_UNIT_TESTS" = "failure" ] || [ "$PCAPKIT_UNIT_TESTS" = "cancelled" ]; then
1278-
echo "::warning title=Release blocked by the test gate::the release test gate concluded '${PCAPKIT_UNIT_TESTS}', so no release job ran for ${PCAPKIT_VERSION}. This is a test failure, not #888."
1279-
exit 0
1280-
fi
1281-
1282-
# #1748: tagged at another commit, with the gate and every publisher
1283-
# skipped for it, which happens only off a v* ref. That release is that
1284-
# commit's (#1634): a notice if complete, else a warning naming what it
1285-
# lacks, both green. A publisher that ran on a v* push meets #1634's
1286-
# check and is reported below.
1287-
if [ "${PCAPKIT_RELEASED_ELSEWHERE:-}" = "true" ] && [ "$PCAPKIT_GITHUB_JOB" = "skipped" ] \
1288-
&& [ "$PCAPKIT_TAG_JOB" = "skipped" ] && [ "$PCAPKIT_PYPI_JOB" = "skipped" ] \
1289-
&& [ "$PCAPKIT_CONDA_JOB" = "skipped" ]; then
1290-
missing=""
1291-
[ "$PCAPKIT_CONDA_TAG_EXISTS" = "true" ] || missing="${missing:+${missing}, }the conda-${PCAPKIT_VERSION}+0 tag"
1292-
[ "$PCAPKIT_PYPI_COMPLETE" = "true" ] || missing="${missing:+${missing}, }some of its PyPI files"
1293-
[ "$PCAPKIT_CONDA_COMPLETE" = "true" ] || missing="${missing:+${missing}, }some of its Anaconda files"
1294-
if [ -z "$missing" ]; then
1295-
echo "::notice title=Nothing to release::v${PCAPKIT_VERSION} names ${PCAPKIT_RELEASED_AT}, not this run's commit ${PCAPKIT_RUN_SHA}, and that release is complete, so this run correctly released nothing (#1748)."
1296-
else
1297-
echo "::warning title=v${PCAPKIT_VERSION} is unfinished (#1634)::v${PCAPKIT_VERSION} names ${PCAPKIT_RELEASED_AT}, not this run's commit ${PCAPKIT_RUN_SHA}, and still lacks ${missing}. Only that commit's run may finish it (#1634), so this run released nothing: re-run the Create Release run created for ${PCAPKIT_RELEASED_AT}, or release the next version."
1298-
fi
1299-
exit 0
1300-
fi
1301-
1302-
# From here the evidence outputs are meaningful. A skipped gate (#1052)
1303-
# falls through on purpose: it skips only when every evidence output
1304-
# reads 'true' off a v* ref, reported as "Nothing to release" below; if
1305-
# its predicate drifts, every publisher skips behind it and this
1306-
# reaches the loud #888 branch instead.
1307-
#
1308-
# Reconcile each job against its OWN evidence, not against each other:
1309-
# a self-heal mixes jobs that skipped because their artefact existed
1310-
# with jobs that ran, and demanding uniformity would read that mix as
1311-
# #888 on the run that finished the release. Reconciled = succeeded, or
1312-
# skipped with its own evidence saying done.
1313-
reconciled_github=false
1314-
[ "$PCAPKIT_GITHUB_JOB" = "success" ] && reconciled_github=true
1315-
[ "$PCAPKIT_GITHUB_JOB" = "skipped" ] && [ "$PCAPKIT_TAG_EXISTS" = "true" ] && reconciled_github=true
1316-
1317-
reconciled_tag=false
1318-
[ "$PCAPKIT_TAG_JOB" = "success" ] && reconciled_tag=true
1319-
[ "$PCAPKIT_TAG_JOB" = "skipped" ] && [ "$PCAPKIT_CONDA_TAG_EXISTS" = "true" ] && reconciled_tag=true
1320-
1321-
reconciled_pypi=false
1322-
[ "$PCAPKIT_PYPI_JOB" = "success" ] && reconciled_pypi=true
1323-
[ "$PCAPKIT_PYPI_JOB" = "skipped" ] && [ "$PCAPKIT_PYPI_COMPLETE" = "true" ] && reconciled_pypi=true
1324-
1325-
reconciled_conda=false
1326-
[ "$PCAPKIT_CONDA_JOB" = "success" ] && reconciled_conda=true
1327-
[ "$PCAPKIT_CONDA_JOB" = "skipped" ] && [ "$PCAPKIT_CONDA_COMPLETE" = "true" ] && reconciled_conda=true
1328-
1329-
all_reconciled=false
1330-
if [ "$reconciled_github" = "true" ] && [ "$reconciled_tag" = "true" ] \
1331-
&& [ "$reconciled_pypi" = "true" ] && [ "$reconciled_conda" = "true" ]; then
1332-
all_reconciled=true
1333-
fi
1334-
1335-
all_skipped=true
1336-
any_broken=false
1337-
for result in "$PCAPKIT_GITHUB_JOB" "$PCAPKIT_TAG_JOB" "$PCAPKIT_PYPI_JOB" "$PCAPKIT_CONDA_JOB"; do
1338-
[ "$result" = "skipped" ] || all_skipped=false
1339-
if [ "$result" = "failure" ] || [ "$result" = "cancelled" ]; then
1340-
any_broken=true
1341-
fi
1342-
done
1343-
1344-
# Case 2: every target reconciled by skipping -- the usual steady state.
1345-
if [ "$all_reconciled" = "true" ] && [ "$all_skipped" = "true" ]; then
1346-
echo "::notice title=Nothing to release::pypcapkit ${PCAPKIT_VERSION} is already fully released -- the v* tag, the conda-*+0 tag and every PyPI and Anaconda file version_check expects are present. This run correctly released nothing."
1347-
exit 0
1348-
fi
1349-
1350-
# Every target reconciled and at least one ran: a fresh release or a
1351-
# self-heal, both quiet.
1352-
if [ "$all_reconciled" = "true" ]; then
1353-
echo "::notice title=Release complete::pypcapkit ${PCAPKIT_VERSION} is now fully released -- github=${PCAPKIT_GITHUB_JOB} tag=${PCAPKIT_TAG_JOB} pypi=${PCAPKIT_PYPI_JOB} conda=${PCAPKIT_CONDA_JOB}. A 'skipped' entry there means that target's own artefact already existed before this run, not that anything was missed."
1354-
exit 0
1355-
fi
1356-
1357-
if [ "$any_broken" = "true" ]; then
1358-
echo "::warning title=Release did not complete::github=${PCAPKIT_GITHUB_JOB} tag=${PCAPKIT_TAG_JOB} pypi=${PCAPKIT_PYPI_JOB} conda=${PCAPKIT_CONDA_JOB} for ${PCAPKIT_VERSION}. At least one release job failed or was cancelled outright -- see its own log; this run's checks already show that failure in red."
1359-
exit 0
1360-
fi
1361-
1362-
# Case 3, the #888 shape: a target skipped despite its own evidence
1363-
# saying incomplete, with nothing failed or cancelled to explain it.
1364-
# The evidence-based `if:`s should make this unreachable, so reaching
1365-
# it means that assumption broke: loud, and never green.
1366-
echo "::error title=Stranded partial release (#888)::${PCAPKIT_VERSION} is only partially published (tag exists=${PCAPKIT_TAG_EXISTS}, conda tag exists=${PCAPKIT_CONDA_TAG_EXISTS}, pypi complete=${PCAPKIT_PYPI_COMPLETE}, conda complete=${PCAPKIT_CONDA_COMPLETE}) but github=${PCAPKIT_GITHUB_JOB} tag=${PCAPKIT_TAG_JOB} pypi=${PCAPKIT_PYPI_JOB} conda=${PCAPKIT_CONDA_JOB} -- at least one target skipped despite its own evidence saying it is not done. Re-running this workflow should let the incomplete jobs self-heal; if this recurs on the next run too, the evidence checks in version_check are the first thing to check."
1367-
exit 1
1237+
run: python3 util/release_status.py

‎docs/source/contributing/workflows.rst‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1267,6 +1267,15 @@ does; each has tests under :file:`tests/project/` that run it.
12671267
* - :file:`util/release_upload.py`
12681268
- ``Upload package to GitHub Release`` in ``create-release.yml``'s ``pypi`` and
12691269
``conda`` (`#1640 <https://github.com/JarryShaw/PyPCAPKit/issues/1640>`__).
1270+
* - :file:`util/release_status.py`
1271+
- ``Report why this run did or did not release anything`` in
1272+
``create-release.yml``'s ``release_status``, on the image's ``python3``
1273+
after a checkout of :file:`util/` alone (`#888
1274+
<https://github.com/JarryShaw/PyPCAPKit/issues/888>`__).
1275+
* - :file:`util/release_vendor_update.py`
1276+
- ``Read whether its vendor-update job ran`` in ``create-release.yml``'s
1277+
``vendor_update``, on the image's ``python3`` after a checkout of
1278+
:file:`util/` alone (`#1748 <https://github.com/JarryShaw/PyPCAPKit/issues/1748>`__).
12701279

12711280
The summaries run on the image's ``/usr/bin/python3`` rather than the leg's
12721281
interpreter, which may have failed to set up or be too broken to run anything, so

‎tests/project/test_actions_helper_2047.py‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -41,8 +41,9 @@
4141
HELPER = ROOT / 'util' / '_actions.py'
4242
#: The scripts that import the helper.
4343
SCRIPTS = ('apt_packages', 'check_suite_report', 'ci_timing', 'classify_changes', 'codeql_report',
44-
'coverage_report', 'leg_summary', 'pip_install', 'regenerate_samples', 'release_tags', 'release_upload',
45-
'update_python_versions', 'vendor_code_changed', 'vendor_test_gate')
44+
'coverage_report', 'leg_summary', 'pip_install', 'regenerate_samples', 'release_status', 'release_tags',
45+
'release_upload', 'release_vendor_update', 'update_python_versions', 'vendor_code_changed',
46+
'vendor_test_gate')
4647
#: The top-level modules the helper may import, each in Python 3.7's standard
4748
#: library: the running interpreter's :data:`sys.stdlib_module_names` would
4849
#: pass ``tomllib``, which 3.7 lacks.
@@ -58,8 +59,10 @@
5859
'leg_summary': (['ktlo'], {'LEG': 'Python 3.8 (KTLO, non-blocking)', 'STEPS': '{}'}),
5960
'pip_install': (['--help'], {}),
6061
'regenerate_samples': (['--help'], {}),
62+
'release_status': (['--help'], {}),
6163
'release_tags': (['--help'], {}),
6264
'release_upload': (['--help'], {}),
65+
'release_vendor_update': (['--help'], {}),
6366
'update_python_versions': (['--help'], {}),
6467
'vendor_code_changed': (['--help'], {}),
6568
'vendor_test_gate': (['--help'], {}),

‎tests/project/test_discussion_report_2030.py‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -602,7 +602,10 @@ def test_create_release_still_reads_vendor_update_alone(self) -> None:
602602
spec = load('cron-vendor.yml')
603603
self.assertNotIn('name', spec['jobs']['vendor-update'])
604604
self.assertEqual([job.get('name', job_id) for job_id, job in spec['jobs'].items()].count('vendor-update'), 1)
605-
release = (WORKFLOWS / 'create-release.yml').read_text(encoding='utf-8')
605+
# #1967: Create Release's vendor_update runs util/release_vendor_update.py, which selects it.
606+
self.assertIn('run: python3 util/release_vendor_update.py',
607+
(WORKFLOWS / 'create-release.yml').read_text(encoding='utf-8'))
608+
release = (ROOT / 'util' / 'release_vendor_update.py').read_text(encoding='utf-8')
606609
self.assertIn('select(.name == "vendor-update")', release)
607610

608611
def test_the_vendor_report_steps_never_fail_the_job(self) -> None:

0 commit comments

Comments
 (0)