Skip to content

Commit e2b4ed3

Browse files
committed
ci(unit-tests): widen the apt bounds so a slow mirror still installs tshark
A ~0.5 MB/s Azure mirror overran the 45s update and 90s install bounds (job 112943258387). Raise them to 60s/180s and both apt step caps from 5 to 10 minutes, so two worst-case attempts (480s) fit with 120s spare. Comment line counts are unchanged so doc line citations still resolve. Closes #1306
1 parent 11ae68e commit e2b4ed3

2 files changed

Lines changed: 26 additions & 25 deletions

File tree

‎.github/workflows/unit-tests.yml‎

Lines changed: 21 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -453,17 +453,17 @@ jobs:
453453
# passed` -- an unrelated mirror turning a green PR red. Stalls in
454454
# `apt-get update` have had the same all-or-nothing shape.
455455
#
456-
# 5 minutes, with one retry inside it. Across 6 runs (120 non-zero
457-
# observations of this step and `pypcap-parity`'s) the slowest was 34s,
458-
# p99 33s, all on PyShark cells. A stall is therefore distinguishable
459-
# from a slow mirror well inside a short per-attempt bound, which makes
460-
# retrying worth it -- 45s for
461-
# `apt-get update` and 90s for `apt-get install`, two attempts, 270s in
462-
# the worst case, under the 5-minute step cap that still holds however
463-
# the hang is caused. The step cap stays well under the 30-minute job
464-
# budget, so a hang reports as this named step timing out rather than as
465-
# an opaque job cancellation. `dpkg --configure -a` before the second
466-
# attempt recovers an install killed mid-configure.
456+
# 10 minutes, with one retry inside it. In 451 samples of this step and
457+
# `pypcap-parity`'s in the 2 days to 2026-10-07, the max was 160-167s, p99
458+
# ~135s. Per #1306 (job 112943258387), a slow mirror overran 45s on
459+
# `apt-get update`, then 90s on `apt-get install` with tshark's 26.5 MB
460+
# of archives part-fetched; passing runs took 78s for 26.5 MB and 56s
461+
# for 27.9 MB. So 60s for update and 180s for install, which fetches
462+
# ~27 MB at down to ~0.15 MB/s; two attempts, 480s
463+
# worst case, 120s under the 10-minute (600s) step cap that holds however
464+
# the hang is caused. That cap stays well under the 30-minute job budget,
465+
# so a hang reports as this named step timing out, not a job cancellation.
466+
# `dpkg --configure -a` before attempt 2 recovers a killed configure.
467467
#
468468
# Deliberately *not* passing `-o Acquire::Retries=3 -o
469469
# Acquire::http::Timeout=30`, which #974 proposed: both are already the
@@ -474,13 +474,13 @@ jobs:
474474
# something apt's per-`select()` data timeout governs, which is why the
475475
# bound here is `timeout(1)` around the whole command.
476476
- name: Install this engine's system packages
477-
timeout-minutes: 5
477+
timeout-minutes: 10
478478
run: |
479479
set -eu
480480
apt_install() {
481481
for attempt in 1 2; do
482-
if sudo timeout 45 apt-get update \
483-
&& sudo DEBIAN_FRONTEND=noninteractive timeout 90 apt-get install -y --no-install-recommends "$@"; then
482+
if sudo timeout 60 apt-get update \
483+
&& sudo DEBIAN_FRONTEND=noninteractive timeout 180 apt-get install -y --no-install-recommends "$@"; then
484484
return 0
485485
fi
486486
echo "::warning title=apt attempt $attempt of 2 failed (${ENGINE})::apt-get update or install failed or exceeded its per-attempt bound; retrying once."
@@ -736,18 +736,18 @@ jobs:
736736
#
737737
# `timeout-minutes` and the retry for the reason given on `engine-tests`'s
738738
# own install step -- see the note there, which this step shares
739-
# wholesale, including the 5-minute cap and per-attempt
740-
# bounds. It was never the leg that stalled, but it installs the union of
741-
# the packages that one does from the same mirrors, so the hazard is
742-
# identical; its observations are in the 120-sample figure quoted there.
739+
# wholesale, including the 10-minute cap (under this job's 45) and the
740+
# 60s/180s bounds. It was never the leg that stalled, but it installs the
741+
# union of the packages that one does from the same mirrors; the hazard is
742+
# identical; its observations are in the 451-sample figure quoted there.
743743
- name: Install libpcap headers, a C toolchain, and tshark
744-
timeout-minutes: 5
744+
timeout-minutes: 10
745745
run: |
746746
set -eu
747747
echo "wireshark-common wireshark-common/install-setuid boolean false" | sudo debconf-set-selections
748748
for attempt in 1 2; do
749-
if sudo timeout 45 apt-get update \
750-
&& sudo DEBIAN_FRONTEND=noninteractive timeout 90 apt-get install -y --no-install-recommends build-essential libpcap-dev tshark; then
749+
if sudo timeout 60 apt-get update \
750+
&& sudo DEBIAN_FRONTEND=noninteractive timeout 180 apt-get install -y --no-install-recommends build-essential libpcap-dev tshark; then
751751
exit 0
752752
fi
753753
echo "::warning title=apt attempt $attempt of 2 failed::apt-get update or install failed or exceeded its per-attempt bound; retrying once."

‎tests/project/test_unit_tests_queue.py‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -458,15 +458,16 @@ def test_two_stalls_fail_after_exactly_two_attempts(self) -> None:
458458
self.assertNotEqual(status, 0)
459459
self.assertEqual(apt, ['update', 'update'])
460460

461-
def test_both_attempts_fit_inside_a_five_minute_step_cap(self) -> None:
462-
"""Every apt call is bounded, and two full attempts fit the 5-minute cap."""
461+
def test_both_attempts_fit_inside_a_ten_minute_step_cap(self) -> None:
462+
"""Every apt call is bounded, and two full attempts fit the 10-minute cap (#1306)."""
463463
for job_name, name, extra in APT_STEPS:
464464
with self.subTest(job=job_name):
465465
status, apt, bounds = self.run_apt(name, job_name, extra, fail_updates=0)
466466
self.assertEqual(status, 0)
467467
self.assertEqual(len(bounds), len(apt), 'an apt-get call runs without timeout')
468-
self.assertEqual(step_cap(job(job_name), name), 5)
469-
self.assertLessEqual(2 * sum(bounds), 5 * 60)
468+
self.assertEqual(bounds, [60, 180])
469+
self.assertEqual(step_cap(job(job_name), name), 10)
470+
self.assertLessEqual(2 * sum(bounds), 10 * 60)
470471

471472

472473
#: Matches a string literal that is a path into docs/ or names Markdown files.

0 commit comments

Comments
 (0)