Repository navigation
Make timing-sensitive CLI tests reliable on a busy host #141
Description
Problem
Four unchanged frontier tests exceeded their five-second limits during the October 15 Cursor update. Cleanup then produced missing-directory and stopped-process errors.
A diagnostic run with 60-second limits passed the frontier tests. A Grok retry test failed because its preflight counter file was absent.
The local load average was 103.65. This observation does not prove the cause. Linux CI passed the full suite with normal limits.
Required work
- Reproduce the failures on a busy host.
- Separate slow test execution from errors caused by cleanup after a timeout.
- Check assumptions about timer order and temporary-file lifetime.
- Reduce fixture cost or correct the timing assumptions.
- Use a larger test limit only when evidence supports that limit.
Do not weaken failure assertions or add worker runtime deadlines.
Completion conditions
The affected tests pass under the reproduced conditions. Failure cases still detect incorrect behavior. Record the conditions and results in the correction PR.
Evidence
The original local logs remain outside the repository in /Users/arjitjaiswal/open-pstack-e5136d8/.
Activity
[-]Stabilize frontier topology fixtures under host contention[/-][+]Stabilize time-sensitive CLI fixtures under host contention[/+][-]Stabilize time-sensitive CLI fixtures under host contention[/-][+]Make timing-sensitive CLI tests reliable on a busy host[/+]arjitj2 commentedon Oct 15, 2026
The local Bun run for PR #147 reported 991 passes and seven failures in unchanged tests. Failures included frontier timeouts, the Grok retry-deadline fixture, and a Devin export fixture. The exact-commit Linux CI passed. PR #147 changes issue text and maintenance report text only. It does not resolve this test problem. This issue remains open. Next, reproduce the timing failures under controlled host load. Evidence: #147 and https://github.com/arjitj2/open-pstack/actions/runs/38769150742.
arjitj2 commentedon Oct 15, 2026
The 1.17.3 catch-up reproduced failures in unchanged tests. See PR #149.
The Node 24 run finished with 985 passes, 13 failures, and 11 additional errors across 998 tests. Failures include frontier tests, provider tests, and wrapper-deadline assertions. Some timeout failures triggered later cleanup errors.
The first run also used a broken Node 25 binary. Both logs remain available under
/Users/arjitjaiswal/open-pstack-99743e9/. No test budgets, test source, or runtime source changed.Linux CI passed 993 tests with five platform-specific skips. All test budgets remained unchanged.
This issue remains open. Reproduce the timing assumptions and fixture cleanup on a busy host. Correct the tests without weakening assertions or adding worker deadlines. PR #149 only changes guidance and does not complete this work.
arjitj2 commentedon Oct 17, 2026
PR #153 reproduced six failures in unchanged tests under Node 24. The run reported 992 passes and five additional errors.
The failures involve frontier discovery, external-provider receipts, and the Grok retry-deadline fixture. Runtime source, test source, and test budgets are unchanged.
Exact-head Linux CI passed 993 tests with five platform skips and zero failures. The local failure cause remains unproven.
This issue remains open. Reproduce the timing assumptions and cleanup behavior, then correct the fixtures without weakening assertions or adding worker deadlines. PR #153 does not resolve this work.
arjitj2 commentedon Oct 17, 2026
The local Node 24 run for #155 reported 990 passes, eight failures, and seven additional errors. Runtime source and test source are unchanged.
Failures include frontier discovery, external-provider receipts, and a Devin environment fixture. The cause remains unproven.
Exact-head Linux CI passed with normal budgets. This issue remains open for fixture reproduction and correction. PR #155 only restores README prose and adds its approval policy.
arjitj2 commentedon Oct 17, 2026
GitHub linked closure to PR #155 because its body contained a closing keyword in a negative sentence. The README correction did not complete the fixture work. The body now uses a non-closing reference. This issue remains open for test reproduction and stabilization.
arjitj2 commentedon Oct 18, 2026
The local Node 24 suite for #158 reported 991 passes, seven failures, and six additional errors. Runtime and test source are unchanged.
Exact-head Linux CI passed 993 tests with five platform skips and zero failures. The local failure cause remains unproven.
This issue remains open for controlled reproduction and fixture stabilization. The TypeScript guidance update in #158 leaves this fixture work separate.