Skip to content

Commit ae80f11

Browse files
authored
ci(python): move the KTLO legs into jobs nothing gating waits on (#1738) (#1746)
- Owner's option A: the KTLO legs of `test`, `integration` and `pypcap-parity` are now `test-ktlo`, `integration-ktlo` and `pypcap-parity-ktlo` in unit-tests.yml. Neither `required-checks` nor `unittest-ordering` needs them, so a lost runner or the job cap on a KTLO leg no longer fails `Required checks passed`. - Each KTLO job runs its twin's steps by YAML alias, so the step-level `continue-on-error`, warning-level annotations and `KTLO summary` are unchanged; job-level `continue-on-error: true` is the backstop. - Check names are unchanged (`Python 3.x (KTLO)` and so on). - util/python_versions.py writes the three KTLO matrices; the gating matrices hold the supported tier only. - util/ci_timing.py resolves aliased steps and picks the most specific job name template; util/update_python_versions.py reports a leg moving between twins as a rename and counts the new jobs' legs. - workflows.rst and releasing.rst describe it; citations renumbered. Tests: test_ktlo_jobs_gate_nothing_1738.py pins the structure; the 1633, 1660, 1672, 1732, apt-cache and queue tests follow the move. Closes #1738
1 parent 0cdd3cd commit ae80f11

14 files changed

Lines changed: 563 additions & 124 deletions

‎.github/workflows/unit-tests.yml‎

Lines changed: 106 additions & 50 deletions
Original file line numberDiff line numberDiff line change
@@ -64,8 +64,11 @@ jobs:
6464
outputs:
6565
latest: ${{ steps.read.outputs.latest }}
6666
test: ${{ steps.read.outputs.test }}
67+
test-ktlo: ${{ steps.read.outputs.test-ktlo }}
6768
integration: ${{ steps.read.outputs.integration }}
69+
integration-ktlo: ${{ steps.read.outputs.integration-ktlo }}
6870
pypcap-parity: ${{ steps.read.outputs.pypcap-parity }}
71+
pypcap-parity-ktlo: ${{ steps.read.outputs.pypcap-parity-ktlo }}
6972
engine-supported: ${{ steps.read.outputs.engine-supported }}
7073
engine-ktlo: ${{ steps.read.outputs.engine-ktlo }}
7174

@@ -89,21 +92,22 @@ jobs:
8992
# this workflow used to) therefore ran the whole matrix in every caller
9093
# and skipped the gate everywhere. An input is the only reliable way for a
9194
# reusable workflow to know how it was invoked.
92-
name: Python ${{ matrix.python-version }}${{ matrix.ktlo && ' (KTLO)' || '' }}
95+
name: Python ${{ matrix.python-version }}
9396
if: ${{ inputs.gate-only != true && needs.changes.outputs.code != 'false' }}
9497
needs: [changes, python-versions]
9598
runs-on: ${{ matrix.os || 'ubuntu-latest' }}
9699
timeout-minutes: 45
97100
# #1672: the level of every annotation a failing step writes -- a warning on
98-
# a KTLO leg, which must not show red, an error on a supported one.
101+
# a KTLO leg, which must not show red, an error on a supported one. The
102+
# steps are shared with `test-ktlo`, whose legs carry `ktlo: true`.
99103
env:
100104
ANNOTATION_LEVEL: ${{ matrix.ktlo && 'warning' || 'error' }}
101105
strategy:
102106
fail-fast: false
103-
# #1660: .github/python-versions.toml's CPython tiers, through the
104-
# `python-versions` job. The supported tier is the `python-version` axis
105-
# and blocks through `required-checks` (#1633); `include` marks one leg
106-
# `coverage: true` and adds each KTLO leg with `ktlo: true`.
107+
# #1660: .github/python-versions.toml's supported CPython tier, through
108+
# the `python-versions` job, as the `python-version` axis; it blocks
109+
# through `required-checks` (#1633). `include` marks one leg
110+
# `coverage: true`. The KTLO legs are `test-ktlo`'s (#1738).
107111
#
108112
# Exactly one leg runs under coverage, so measuring it adds no leg to
109113
# the queue. It is `latest`, the newest supported version, like every
@@ -112,32 +116,12 @@ jobs:
112116
# coverage cost 1.8x on 3.15 locally (2.0x on 3.14): a step of about
113117
# 15-19 minutes under the job's 45.
114118
#
115-
# The KTLO tier (#1633): end-of-life versions that still build and
116-
# test but never block. `required-checks` reads this job's one
117-
# `needs.test.result`, so a KTLO leg cannot fail the job: each step
118-
# that can fail on one carries `continue-on-error: ${{ matrix.ktlo
119-
# == true }}` instead -- the step-level form, whose effect on the job
120-
# is documented, as for engine-tests' KTLO cell. The leg concludes
121-
# success, and the job's last step, `KTLO summary`, reports each
122-
# failed step as a warning (#1672). A version `ubuntu-latest` cannot
123-
# provide carries its `runner` from the file as `os`: 3.8 and 3.9 need
124-
# ubuntu-24.04, as setup-python has no Ubuntu 26.04 build of either
125-
# and `ubuntu-latest` moves to 26.04 from 2026-10-19.
126-
#
127-
# 3.7 needs ubuntu-22.04, the newest image setup-python builds it for
128-
# (3.7.17). It cannot install today -- the `test` extra's `pytest>=8`
129-
# needs 3.8, and the sources use syntax 3.7 cannot parse -- so the leg
130-
# stops at its install step, with one annotation naming pip's reason
131-
# (see that step). ubuntu-22.04 is deprecated: brownouts from
132-
# 2027-03-23, retirement on 2027-04-17 (actions/runner-images#14254).
133-
# A brownout fails the leg before any step runs, which no step-level
134-
# `continue-on-error` can contain, so this leg must go before then.
135-
#
136119
# The experimental tier (#1660) runs in experimental.yml, a workflow of
137120
# its own, so that nothing here waits on it (#1732).
138121
matrix: ${{ fromJSON(needs.python-versions.outputs.test) }}
139122

140-
steps:
123+
# `&test-steps`: `test-ktlo` runs these same steps, by YAML alias.
124+
steps: &test-steps
141125
- uses: actions/checkout@v7
142126
id: checkout
143127
continue-on-error: ${{ matrix.ktlo == true }}
@@ -419,8 +403,47 @@ jobs:
419403
echo "- ${LEG}: step \`${id}\` failed; non-blocking, so this job still succeeds." >> "$GITHUB_STEP_SUMMARY"
420404
done <<< "$failed"
421405
406+
# The KTLO tier (#1633): end-of-life versions that still build and test but
407+
# never block. #1738, the owner's option A: their legs are this job's, not
408+
# `test`'s, and neither `required-checks` nor `unittest-ordering` `needs:`
409+
# it, so a failure no step can contain -- a lost runner, the job cap --
410+
# fails nothing that gates. The steps are `test`'s, by YAML alias
411+
# (docs.github.com, Reusing workflow configurations, "YAML anchors and
412+
# aliases"), and each that can fail on a KTLO leg carries
413+
# `continue-on-error: ${{ matrix.ktlo == true }}`, so the leg concludes
414+
# success, its annotations are warnings, and `KTLO summary`, the last step,
415+
# reports each failed step as a warning (#1672). The job-level
416+
# `continue-on-error` is the backstop for the rest: it "Prevents a workflow
417+
# run from failing when a job fails" (docs.github.com, Workflow syntax,
418+
# `jobs.<job_id>.continue-on-error`), as on experimental.yml's job, so the
419+
# Unit Tests run its `workflow_run` consumers read stays green.
420+
#
421+
# A version `ubuntu-latest` cannot provide carries its `runner` from the
422+
# file as `os`: 3.8 and 3.9 need ubuntu-24.04, as setup-python has no Ubuntu
423+
# 26.04 build of either and `ubuntu-latest` moves to 26.04 from 2026-10-19.
424+
# 3.7 needs ubuntu-22.04, the newest image setup-python builds it for
425+
# (3.7.17). It cannot install today -- the `test` extra's `pytest>=8` needs
426+
# 3.8, and the sources use syntax 3.7 cannot parse -- so the leg stops at
427+
# its install step, with one annotation naming pip's reason (see that step).
428+
# ubuntu-22.04 is deprecated: brownouts from 2027-03-23, retirement on
429+
# 2027-04-17 (actions/runner-images#14254); a brownout fails the leg before
430+
# any step runs, which only the job-level guard contains.
431+
test-ktlo:
432+
name: Python ${{ matrix.python-version }} (KTLO)
433+
if: ${{ inputs.gate-only != true && needs.changes.outputs.code != 'false' }}
434+
needs: [changes, python-versions]
435+
runs-on: ${{ matrix.os || 'ubuntu-latest' }}
436+
timeout-minutes: 45
437+
continue-on-error: true
438+
env:
439+
ANNOTATION_LEVEL: ${{ matrix.ktlo && 'warning' || 'error' }}
440+
strategy:
441+
fail-fast: false
442+
matrix: ${{ fromJSON(needs.python-versions.outputs.test-ktlo) }}
443+
steps: *test-steps
444+
422445
integration:
423-
name: Integration Python ${{ matrix.python-version }}${{ matrix.ktlo && ' (KTLO)' || '' }}
446+
name: Integration Python ${{ matrix.python-version }}
424447
if: ${{ inputs.gate-only != true && needs.changes.outputs.code != 'false' }}
425448
needs: [changes, python-versions]
426449
runs-on: ${{ matrix.os || 'ubuntu-latest' }}
@@ -430,11 +453,12 @@ jobs:
430453
ANNOTATION_LEVEL: ${{ matrix.ktlo && 'warning' || 'error' }}
431454
strategy:
432455
fail-fast: false
433-
# The two tiers of the `test` job above, for the same reasons, without
434-
# its coverage leg.
456+
# The supported tier of the `test` job above, without its coverage leg;
457+
# the KTLO legs are `integration-ktlo`'s (#1738).
435458
matrix: ${{ fromJSON(needs.python-versions.outputs.integration) }}
436459

437-
steps:
460+
# `&integration-steps`: `integration-ktlo` runs these same steps.
461+
steps: &integration-steps
438462
- uses: actions/checkout@v7
439463
id: checkout
440464
continue-on-error: ${{ matrix.ktlo == true }}
@@ -610,6 +634,21 @@ jobs:
610634
echo "- ${LEG}: step \`${id}\` failed; non-blocking, so this job still succeeds." >> "$GITHUB_STEP_SUMMARY"
611635
done <<< "$failed"
612636
637+
# `integration`'s KTLO legs, apart for the reasons on `test-ktlo` (#1738).
638+
integration-ktlo:
639+
name: Integration Python ${{ matrix.python-version }} (KTLO)
640+
if: ${{ inputs.gate-only != true && needs.changes.outputs.code != 'false' }}
641+
needs: [changes, python-versions]
642+
runs-on: ${{ matrix.os || 'ubuntu-latest' }}
643+
timeout-minutes: 45
644+
continue-on-error: true
645+
env:
646+
ANNOTATION_LEVEL: ${{ matrix.ktlo && 'warning' || 'error' }}
647+
strategy:
648+
fail-fast: false
649+
matrix: ${{ fromJSON(needs.python-versions.outputs.integration-ktlo) }}
650+
steps: *integration-steps
651+
613652
# Per-engine coverage for the third-party capture engines (#845): every
614653
# (Python version, engine) pair is checked, and a failure -- or a correctly
615654
# asserted decline -- names the one engine and interpreter responsible. Each
@@ -1102,7 +1141,7 @@ jobs:
11021141
# step-level `continue-on-error`, for the reason given on the `test` job's
11031142
# matrix.
11041143
pypcap-parity:
1105-
name: PyPCAP/PyPCAPFile parity Python ${{ matrix.python-version }}${{ matrix.ktlo && ' (KTLO)' || '' }}
1144+
name: PyPCAP/PyPCAPFile parity Python ${{ matrix.python-version }}
11061145
if: ${{ inputs.gate-only != true && needs.changes.outputs.code != 'false' }}
11071146
needs: [changes, python-versions]
11081147
runs-on: ubuntu-latest
@@ -1121,7 +1160,8 @@ jobs:
11211160
fail-fast: false
11221161
matrix: ${{ fromJSON(needs.python-versions.outputs.pypcap-parity) }}
11231162

1124-
steps:
1163+
# `&pypcap-parity-steps`: `pypcap-parity-ktlo` runs these same steps.
1164+
steps: &pypcap-parity-steps
11251165
- uses: actions/checkout@v7
11261166
id: checkout
11271167
continue-on-error: ${{ matrix.ktlo == true }}
@@ -1333,6 +1373,25 @@ jobs:
13331373
echo "- ${LEG}: step \`${id}\` failed; non-blocking, so this job still succeeds." >> "$GITHUB_STEP_SUMMARY"
13341374
done <<< "$failed"
13351375
1376+
# `pypcap-parity`'s KTLO legs, apart for the reasons on `test-ktlo` (#1738).
1377+
# Same apt set and cache key; `pypcap-parity`'s supported leg saves it.
1378+
pypcap-parity-ktlo:
1379+
name: PyPCAP/PyPCAPFile parity Python ${{ matrix.python-version }} (KTLO)
1380+
if: ${{ inputs.gate-only != true && needs.changes.outputs.code != 'false' }}
1381+
needs: [changes, python-versions]
1382+
runs-on: ubuntu-latest
1383+
timeout-minutes: 45
1384+
continue-on-error: true
1385+
env:
1386+
APT_PACKAGES: build-essential libpcap-dev tshark
1387+
APT_SET: pypcap-parity
1388+
APT_CACHE_SAVE: 'false'
1389+
ANNOTATION_LEVEL: ${{ matrix.ktlo && 'warning' || 'error' }}
1390+
strategy:
1391+
fail-fast: false
1392+
matrix: ${{ fromJSON(needs.python-versions.outputs.pypcap-parity-ktlo) }}
1393+
steps: *pypcap-parity-steps
1394+
13361395
# GitHub issue #981: two test modules
13371396
# (tests/protocols/application/test_http_unit.py and
13381397
# tests/test_base_class_contract.py) each pass alone, and pytest -- the
@@ -1550,9 +1609,9 @@ jobs:
15501609
# to the GitHub Release body: a drifted copy is not merely wrong in the tree, it
15511610
# is published.
15521611
#
1553-
# Deliberately *not* gated on ``gate-only``, unlike the four jobs above
1554-
# (`test`, `integration`, `engine-tests`, `pypcap-parity`), and that is the
1555-
# point of putting it here at all. ``create-release.yml`` calls this
1612+
# Deliberately *not* gated on ``gate-only``, unlike the matrix jobs above
1613+
# (`test`, `integration`, `engine-tests`, `pypcap-parity` and the KTLO ones),
1614+
# and that is the point of putting it here at all. ``create-release.yml`` calls this
15561615
# workflow as its release gate, so an ungated job runs on the release path and
15571616
# the release body cannot be built from a file that has drifted. The matrix is
15581617
# skipped per caller because it is expensive and already ran for the commit;
@@ -1850,19 +1909,16 @@ jobs:
18501909
# here: the cell script exits 0 for a cell that declined exactly as
18511910
# expected, so they contribute to that job's `success` like any other cell.
18521911
#
1853-
# Nor is the KTLO tier (#1633): engine-tests' 3.10 cell, and the 3.7-3.10
1854-
# legs of `test` and `integration` and the 3.10 leg of `pypcap-parity`.
1855-
# Each is non-blocking through step-level `continue-on-error`, whose effect
1856-
# on the job's result *is* documented (a step's `conclusion` is its result
1857-
# after `continue-on-error` is applied), so the job, and this gate, read
1858-
# `success` while the failed step stays visible. Not a job-level
1859-
# `continue-on-error`: GitHub does not document whether `needs.<job>.result`
1860-
# sees such a job's failure as `failure` or `success`, so a KTLO-only
1861-
# regression could block every merge. The one failure no step can contain is
1862-
# a leg that never gets a runner: the 3.7 legs' deprecated ubuntu-22.04 image
1863-
# has brownouts from 2027-03-23 -- see the `test` job's matrix. The
1864-
# experimental legs (#1660) are in experimental.yml, which nothing here
1865-
# needs or waits on (#1732).
1912+
# Nor is the KTLO tier (#1633). engine-tests' KTLO cell is non-blocking
1913+
# through step-level `continue-on-error`, whose effect on the job's result
1914+
# *is* documented (a step's `conclusion` is its result after
1915+
# `continue-on-error` is applied), so the job, and this gate, read `success`
1916+
# while the failed step stays visible. The KTLO legs of `test`,
1917+
# `integration` and `pypcap-parity` are `test-ktlo`, `integration-ktlo`
1918+
# and `pypcap-parity-ktlo` (#1738), which this job does not need, so not
1919+
# even a failure no step can contain -- a lost runner, the job cap -- can
1920+
# fail it. The experimental legs (#1660) are in experimental.yml, which
1921+
# nothing here needs or waits on (#1732).
18661922
#
18671923
# The docs-only path. On a pull request whose diff touches nothing but
18681924
# `docs/**` and `*.md` files outside the code directories (the `changes`

‎docs/source/contributing/releasing.rst‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ Editing ``__version__`` by hand without also moving :file:`CITATION.cff`'s
5252
(``:494-514``) asserts the two agree, and ``create-release.yml``'s
5353
``unit-tests`` job (``:467-481``) calls ``unit-tests.yml`` with
5454
``gate-only: true``, which runs the **full** suite rather than the tiered
55-
subset an ordinary push runs (its ``gate`` job, ``unit-tests.yml:1717-1787``).
55+
subset an ordinary push runs (its ``gate`` job, ``unit-tests.yml:1776-1846``).
5656
That gate runs whenever a release will -- a moved ``__version__`` is exactly
5757
what makes ``version_check``'s evidence read "not yet published" -- and every
5858
publishing job requires it to have succeeded (``:500,685,850,1085``), so a
@@ -62,7 +62,7 @@ Either run the script, or move both fields by hand in the same commit.
6262
This is the **only** edit a person makes to get a release started -- the tag,
6363
the Release, and every upload are the workflow's job from here. Two other
6464
checks run unconditionally on the same path, though neither is a step in *this*
65-
process: the ``changelog`` job (``unit-tests.yml:1565-1584``) fails outright if
65+
process: the ``changelog`` job (``unit-tests.yml:1624-1643``) fails outright if
6666
``CHANGELOG.md`` has drifted from its source entry under
6767
:file:`docs/source/changelog/`, and the release-body step warns, without
6868
failing, if that entry's heading still reads "unreleased"

0 commit comments

Comments
 (0)