Skip to content

Commit 4bd0be4

Browse files
committed
ci: install through util/pip_install.py in experimental and Compat (#1967)
- experimental.yml: the experimental job's install step runs the script in place of its inline shell; same extras, id, if, continue-on-error, timeout, shell and LEG, at the job's warning level. - python-compatibility.yml: Compat's install step runs the script, with LEG on the step, so a failed install now writes the job's ANNOTATION_LEVEL annotation; id, if, continue-on-error unchanged. - tests: test_pip_install_1967.py pins both steps whole, their jobs' levels, and every other pip install in those workflows; the 1633 install tests run Compat's step too, under the default shell. - workflows.rst: the script's callers include both jobs.
1 parent 7c5f371 commit 4bd0be4

6 files changed

Lines changed: 101 additions & 29 deletions

File tree

‎.github/workflows/experimental.yml‎

Lines changed: 3 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -231,7 +231,8 @@ jobs:
231231
python-version: ${{ matrix.python-version }}
232232
cache: pip
233233

234-
# As unit-tests.yml `test`'s install, for the same reasons.
234+
# As unit-tests.yml `test`'s install, for the same reasons: util/pip_install.py
235+
# annotates a failed install, at the job's `warning` (#1967).
235236
- name: Install package and test dependencies
236237
id: install
237238
if: ${{ steps.python.outcome == 'success' }}
@@ -240,14 +241,7 @@ jobs:
240241
shell: bash
241242
env:
242243
LEG: Python ${{ matrix.python-version }} (experimental, non-blocking)
243-
run: |
244-
python -m pip install -U pip setuptools wheel
245-
if ! python -m pip install -e '.[test,DPKT,crypto,NGAP]' 2>&1 | tee "$RUNNER_TEMP/pip-install.log"; then
246-
reason=$(sed -n 's/^ERROR: //p' "$RUNNER_TEMP/pip-install.log" | tail -n 1)
247-
if [ -z "$reason" ]; then reason="pip failed without an ERROR line; see this step's log"; fi
248-
echo "::${ANNOTATION_LEVEL} title=pcapkit did not install on ${LEG//,/%2C}::${reason//%/%25}. The rest of this leg is skipped."
249-
exit 1
250-
fi
244+
run: python util/pip_install.py -e '.[test,DPKT,crypto,NGAP]'
251245

252246
- name: Report available parallelism
253247
id: parallelism

‎.github/workflows/python-compatibility.yml‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -78,13 +78,14 @@ jobs:
7878
# The install needs this leg's interpreter, and the check below needs the
7979
# install, as in unit-tests.yml's `test` job: on a KTLO leg a failed step
8080
# does not stop the job, so each one waits on the `outcome` it needs.
81+
# util/pip_install.py annotates a failed install, as in `test` (#1967).
8182
- name: Install package
8283
id: install
8384
if: ${{ steps.python.outcome == 'success' }}
8485
continue-on-error: ${{ matrix.ktlo == true }}
85-
run: |
86-
python -m pip install -U pip setuptools wheel
87-
python -m pip install -e .
86+
env:
87+
LEG: Python ${{ matrix.python-version }}${{ matrix.ktlo && ' (KTLO, non-blocking)' || '' }}
88+
run: python util/pip_install.py -e .
8889

8990
# One annotation saying why, so a failed leg -- such as 3.7's, where the
9091
# install succeeds and the sources do not compile -- is legible from the

‎docs/source/contributing/workflows.rst‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1209,8 +1209,10 @@ does; each has tests under :file:`tests/project/` that run it.
12091209
and ``gate``, at ``error``.
12101210
* - :file:`util/pip_install.py`
12111211
- The install step of ``unit-tests.yml``'s ``test``, ``integration`` and
1212-
``pypcap-parity`` (and so their KTLO twins), at the job's ``ANNOTATION_LEVEL``,
1213-
and of ``unittest-ordering``, ``project-tests`` and ``gate``, at ``error``.
1212+
``pypcap-parity`` (and so their KTLO twins), ``experimental.yml``'s
1213+
``experimental`` and ``python-compatibility.yml``'s ``compatibility``, at the
1214+
job's ``ANNOTATION_LEVEL``, and of ``unittest-ordering``, ``project-tests`` and
1215+
``gate``, at ``error``.
12141216

12151217
The summaries run on the image's ``/usr/bin/python3`` rather than the leg's
12161218
interpreter, which may have failed to set up or be too broken to run anything, so
@@ -1222,7 +1224,7 @@ file-size cap every test command of ``unit-tests.yml`` takes.
12221224
:file:`util/regenerate_samples.py` is standard library only but runs on the leg's
12231225
interpreter, since the generator it runs imports ``pcapkit``, and so does
12241226
:file:`util/pip_install.py`, since it installs into that interpreter; it runs on
1225-
every version the matrices hold, 3.7 included. Both ``changes`` jobs check out on every event, since the script is in the
1227+
every version the matrices hold, 3.7 and the experimental tier included. Both ``changes`` jobs check out on every event, since the script is in the
12261228
tree. ``create-release.yml``'s ``pypi`` job keeps its own copy of
12271229
the KTLO summary's shell until its shell moves to scripts too; a project pin runs
12281230
that copy and the script on the same steps and requires the same output.

‎tests/project/test_pip_install_1967.py‎

Lines changed: 72 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
# -*- coding: utf-8 -*-
2-
"""#1967: :file:`util/pip_install.py`, the install step of most ``unit-tests.yml`` jobs, run as the steps run it.
2+
"""#1967: :file:`util/pip_install.py`, the install step of most test jobs, run as the steps run it.
33
44
* :class:`TestPipInstall` runs the script against a stand-in ``pip``: a clean
55
install, a failed one with and without an ``ERROR:`` line, a failed upgrade,
@@ -11,6 +11,9 @@
1111
* :class:`TestTheStepsRunTheScript` pins each covered step to the script, with
1212
its guards and environment, and every other ``pip install`` in
1313
``unit-tests.yml`` to a listed step that keeps its shell.
14+
* :class:`TestTheOtherWorkflowsRunTheScript` does the same for the install of
15+
``experimental.yml``'s ``experimental`` and ``python-compatibility.yml``'s
16+
``compatibility``.
1417
1518
"""
1619

@@ -29,11 +32,11 @@
2932
import unittest
3033

3134
from tests._dependency_gates import EDITABLE_INSTALL
32-
from tests._workflows import job, load, steps
35+
from tests._workflows import WORKFLOWS, job, load, steps
3336

3437
ROOT = pathlib.Path(__file__).resolve().parents[2]
3538
SCRIPT = ROOT / 'util' / 'pip_install.py'
36-
WORKFLOW = ROOT / '.github' / 'workflows' / 'unit-tests.yml'
39+
WORKFLOW = WORKFLOWS / 'unit-tests.yml'
3740
LOG = 'pip-install.log'
3841
SPEC = '.[test,DPKT]'
3942

@@ -72,6 +75,36 @@
7275
TWINS = {'test-ktlo': 'test', 'integration-ktlo': 'integration', 'pypcap-parity-ktlo': 'pypcap-parity'}
7376
#: Text only the former inline install held.
7477
INLINE = ('pip-install.log', 'did not install on', "s/^ERROR: //p")
78+
#: The covered steps outside ``unit-tests.yml``, as ``(workflow, job)``, each step whole,
79+
#: and its job's ``ANNOTATION_LEVEL``. The experimental legs are never blocking, so
80+
#: always ``warning``; Compat's KTLO legs take ``warning`` as ``test``'s do. Compat
81+
#: installs no extra, and its step keeps the default shell.
82+
ELSEWHERE = {
83+
('experimental.yml', 'experimental'): ({
84+
'name': 'Install package and test dependencies',
85+
'id': 'install',
86+
'if': "${{ steps.python.outcome == 'success' }}",
87+
'continue-on-error': True,
88+
'timeout-minutes': 10,
89+
'shell': 'bash',
90+
'env': {'LEG': 'Python ${{ matrix.python-version }} (experimental, non-blocking)'},
91+
'run': "python util/pip_install.py -e '.[test,DPKT,crypto,NGAP]'",
92+
}, 'warning'),
93+
('python-compatibility.yml', 'compatibility'): ({
94+
'name': 'Install package',
95+
'id': 'install',
96+
'if': "${{ steps.python.outcome == 'success' }}",
97+
'continue-on-error': '${{ matrix.ktlo == true }}',
98+
'env': {'LEG': MATRIX_LEG},
99+
'run': 'python util/pip_install.py -e .',
100+
}, MATRIX_LEVEL),
101+
}
102+
#: The other ``pip install`` steps of those workflows, which install ``requests`` for
103+
#: util/superseded.py on an interpreter of their own, never the package.
104+
ELSEWHERE_INLINE = frozenset({
105+
('experimental.yml', 'current', 'Install requests'),
106+
('experimental.yml', 'experimental', "Install the watcher's requests"),
107+
})
75108

76109
#: A stand-in ``pip``: it appends its arguments to ``calls``, then does what
77110
#: ``PIP_<STEP>`` says, ``STEP`` ``UPGRADE`` or ``INSTALL``: ``OUTPUT`` lines, each
@@ -368,5 +401,41 @@ def test_the_listed_steps_still_install(self) -> None:
368401
self.assertIn('pip install', str(self.step(job_id, name)['run']))
369402

370403

404+
class TestTheOtherWorkflowsRunTheScript(unittest.TestCase):
405+
"""The experimental and Compat installs run the script, guards unchanged; nothing else there keeps the shell."""
406+
407+
def test_each_site_is_the_script_with_its_guards(self) -> None:
408+
for (name, job_id), (expected, _) in ELSEWHERE.items():
409+
with self.subTest(workflow=name, job=job_id):
410+
found = [entry for entry in steps(WORKFLOWS / name, job_id) if entry.get('name') == expected['name']]
411+
self.assertEqual(found, [expected])
412+
413+
def test_each_job_sets_the_level(self) -> None:
414+
for (name, job_id), (_, level) in ELSEWHERE.items():
415+
with self.subTest(workflow=name, job=job_id):
416+
self.assertEqual(job(WORKFLOWS / name, job_id)['env']['ANNOTATION_LEVEL'], level)
417+
418+
def test_every_other_pip_install_is_a_listed_one(self) -> None:
419+
sites = {(name, job_id, expected['name']) for (name, job_id), (expected, _) in ELSEWHERE.items()}
420+
found = []
421+
for name in sorted({name for name, _ in ELSEWHERE}):
422+
for job_id, body in load(WORKFLOWS / name)['jobs'].items():
423+
for step in body.get('steps', []):
424+
where = (name, job_id, step.get('name'))
425+
run = str(step.get('run', ''))
426+
if re.search(r'pip install|pip_install\.py', run) and where not in sites | ELSEWHERE_INLINE:
427+
found.append(f'{name} {job_id} {step.get("name")!r}')
428+
found.extend(f'{name} {job_id} {step.get("name")!r}: {piece}' for piece in INLINE if piece in run)
429+
self.assertEqual(found, [])
430+
431+
def test_the_listed_steps_still_install(self) -> None:
432+
"""So :data:`ELSEWHERE_INLINE` names nothing stale."""
433+
for name, job_id, step_name in ELSEWHERE_INLINE:
434+
with self.subTest(workflow=name, job=job_id, step=step_name):
435+
found = [entry for entry in steps(WORKFLOWS / name, job_id) if entry.get('name') == step_name]
436+
self.assertEqual(len(found), 1)
437+
self.assertIn('pip install', str(found[0]['run']))
438+
439+
371440
if __name__ == '__main__':
372441
unittest.main()

‎tests/project/test_python_matrix_tiers_1633.py‎

Lines changed: 12 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -509,6 +509,8 @@ def test_no_conda_leg_is_ktlo_and_315_leaves_base_python_alone(self) -> None:
509509
('unit-tests.yml', 'pypcap-parity', 'Install package, test and generator dependencies'),
510510
('python-compatibility.yml', 'compatibility', 'Install package'),
511511
)
512+
#: The install steps of the 3.7 legs, each running util/pip_install.py (#1967).
513+
INSTALLS_37 = (LEGACY_37[0], LEGACY_37[1], INSTALLS[3])
512514

513515

514516
def workflow_commands(output: 'str') -> 'list[str]':
@@ -573,17 +575,19 @@ def setUp(self) -> None:
573575
def run_step(self, name: 'str', job_id: 'str', step_name: 'str', stub: 'str',
574576
env: 'dict[str, str]', leg: 'Optional[dict[str, Any]]' = None,
575577
) -> 'subprocess.CompletedProcess[str]':
576-
"""Step ``step_name`` on ``leg`` (3.7's by default) under ``shell: bash``, ``python`` stubbed.
578+
"""Step ``step_name`` on ``leg`` (3.7's by default) under its shell, ``python`` stubbed.
577579
578580
``LEG`` and the job's ``ANNOTATION_LEVEL`` get the values their expressions
579-
take on that leg, and ``-eo pipefail`` is what ``shell: bash`` runs. With
580-
:data:`PIP_STUB`, ``python`` is this interpreter and the stub is its ``pip``.
581+
take on that leg; ``-eo pipefail`` is what ``shell: bash`` runs, and ``-e``
582+
the default shell. With :data:`PIP_STUB`, ``python`` is this interpreter and
583+
the stub is its ``pip``.
581584
582585
"""
583586
leg = KTLO_37 if leg is None else leg
584587
spec = job(name, job_id)
585588
step = next(step for step in spec['steps'] if step.get('name') == step_name)
586-
self.assertEqual(step.get('shell'), 'bash')
589+
self.assertIn(step.get('shell'), ('bash', None))
590+
flags = ['--noprofile', '--norc', '-eo', 'pipefail'] if step.get('shell') == 'bash' else ['-e']
587591
self.assertEqual(step['env']['LEG'], leg_template(name, job_id))
588592
self.assertEqual(spec['env']['ANNOTATION_LEVEL'], annotation_level(name, job_id))
589593
env = dict(env, LEG=leg_label(leg), ANNOTATION_LEVEL='warning' if is_nonblocking(leg) else 'error')
@@ -601,7 +605,7 @@ def run_step(self, name: 'str', job_id: 'str', step_name: 'str', stub: 'str',
601605
else:
602606
python.write_text('#!/bin/bash\n' + stub, encoding='utf-8')
603607
python.chmod(0o755)
604-
return subprocess.run(['bash', '--noprofile', '--norc', '-eo', 'pipefail', '-c', step['run']],
608+
return subprocess.run(['bash', *flags, '-c', step['run']],
605609
env=dict(os.environ, PATH=f'{bin_dir}{os.pathsep}{os.environ["PATH"]}',
606610
RUNNER_TEMP=str(self.tmp), **env),
607611
cwd=ROOT, capture_output=True, text=True, check=False, timeout=60)
@@ -648,7 +652,7 @@ def test_a_leg_that_cannot_install_stops_there(self) -> None:
648652
self.assertIn("steps.install.outcome == 'success'", condition)
649653

650654
def test_a_failed_install_fails_once_naming_pips_reason(self) -> None:
651-
for name, job_id, install in LEGACY_37[:2]:
655+
for name, job_id, install in INSTALLS_37:
652656
with self.subTest(job=job_id):
653657
result = self.run_step(name, job_id, install, PIP_STUB, {'PIP_OUTPUT': PIP_37, 'PIP_STATUS': '1'})
654658
self.assertNotEqual(result.returncode, 0)
@@ -670,7 +674,7 @@ def test_an_install_failure_without_an_error_line_is_still_named(self) -> None:
670674
'The rest of this leg is skipped.'])
671675

672676
def test_a_clean_install_passes_without_an_annotation(self) -> None:
673-
for name, job_id, install in LEGACY_37[:2]:
677+
for name, job_id, install in INSTALLS_37:
674678
with self.subTest(job=job_id):
675679
result = self.run_step(name, job_id, install, PIP_STUB,
676680
{'PIP_OUTPUT': 'Successfully installed pypcapkit-1.5.0', 'PIP_STATUS': '0'})
@@ -702,7 +706,7 @@ def test_a_supported_leg_fails_the_same_steps_at_error_level(self) -> None:
702706
cases = [(name, job_id, install, PIP_STUB, {'PIP_OUTPUT': PIP_37, 'PIP_STATUS': '1'},
703707
'::error title=pcapkit did not install on Python 3.12::No matching distribution found '
704708
'for pytest>=8; extra == "test". The rest of this leg is skipped.')
705-
for name, job_id, install in LEGACY_37[:2]]
709+
for name, job_id, install in INSTALLS_37]
706710
cases.append(('python-compatibility.yml', 'compatibility', 'Verify package', VERIFY_STUB,
707711
{'COMPILE_OUTPUT': '', 'COMPILE_STATUS': '0', 'IMPORT_STATUS': '1'},
708712
"::error title=pcapkit does not import on Python 3.12::See the traceback in this step's log."))

‎util/pip_install.py‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,10 @@
33
44
The install step of each :file:`.github/workflows/unit-tests.yml` job that installs
55
the package -- ``test``, ``integration``, ``pypcap-parity`` (and so their KTLO
6-
twins), ``unittest-ordering``, ``project-tests`` and ``gate`` -- runs, from the
7-
checkout's root and on the leg's own interpreter::
6+
twins), ``unittest-ordering``, ``project-tests`` and ``gate`` -- and of the
7+
``experimental`` job of :file:`.github/workflows/experimental.yml` and the
8+
``compatibility`` job of :file:`.github/workflows/python-compatibility.yml` runs,
9+
from the checkout's root and on the leg's own interpreter::
810
911
python util/pip_install.py -e '.[test,DPKT,crypto,NGAP]'
1012
@@ -27,7 +29,7 @@
2729
so every later step, which waits on the install's ``outcome``, is skipped. An
2830
``ANNOTATION_LEVEL`` other than ``error`` or ``warning`` exits 2 before pip runs.
2931
The script is standard library only, and runs on every interpreter the matrices
30-
hold, 3.7 included.
32+
hold, 3.7 and the experimental tier included.
3133
3234
"""
3335

0 commit comments

Comments
 (0)