Skip to content

test: 86 timeouts still sit at 120s, and an 85.2s case is about one bad runner draw from red #204

Description

@wormeyman

Follow-up to #203, which fixed one file and deliberately left the rest.

What #203 established

previewAgreement.spec.ts's Vulcanus rock and cliff coverage test failed CI with Error: Test timed out in 120000ms at 150.5s, on a docs-only PR. Its own work had not changed. Across five consecutive runs:

tree duration result
main, before the scrap work 69.6s pass
#202 90.1s pass
main, after #202 merged 108.8s pass
#203 150.5s timed out
#203, at the new 300s budget 139.7s pass

That file now uses }, 300000). Nothing else was touched.

Why this is not finished

86 annotations across 29 files still carry }, 120000), and the ceiling is per-test and hand-written, so a shard rebalance changes which tests sit near it.

The nearest to the edge is vulcanusCliffRejectionStage.spec.ts:

the post-filter model predicts 1,662 survivor-keeps-edge cases and the game shows 0 - 85.2s

Run-to-run spread on these runners measures about 40%, which was established directly rather than inferred: #202's run and #203's run share the same 210 spec files, so vitest hands them identical shards, and the jobs still came in at 259 / 351 / 389 / 133 against 366 / 273 / 327 / 137 - shard 1 up 41%, shard 3 down 16%, and the binding shard changing identity from 3 to 1.

85.2s x 1.4 is 119s. That is inside the ceiling by under a second.

Options, in the order they were weighed

  1. Sweep all 86 to 300000. 29 files, mechanical, removes the class of failure. Costs a large diff, and slows a genuine hang to 5 minutes before it reports.
  2. Raise only the files with a case above ~60s. Smaller diff, same protection where it matters, but needs the per-test scan re-run whenever the suite grows.
  3. Delete the annotations and raise the global testTimeout from 30s. One definition instead of 91, which is how CLAUDE.md frames why the annotations exist at all. Largest behavioural change, and it weakens the budget for every fast test too.

Deliberately not done in #203, which was a docs PR. No action is needed until a shard goes red on a timeout; CLAUDE.md's testTimeout section already names this file as the first place to look when one does.

What is NOT the fix

Retries are banned here and the reason still holds - nothing in this suite is nondeterministic, so a retry would only hide a real regression. And splitting the heavy file is not it either: #203 measured that and recorded why in CLAUDE.md's CI section.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    toolchain-ciBuild, test, CI, dependencies, developer tooling

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions