Skip to content

test(workflow): remove timing flake from rate-limit burst and refill test - #7433

Merged
masenf merged 1 commit into
mainfrom
codex/workflow-rate-test-flake
Oct 9, 2026
Merged

masenf merged 1 commit into
mainfrom
codex/workflow-rate-test-flake

Conversation

@masenf

@masenf masenf commented Oct 6, 2026 •

Copy link
Copy Markdown
Collaborator

The PostgreSQL workflow job failed because test_a_rate_limit_lets_through_a_burst_then_refills measured the fourth workflow start from before sequential database inserts and required it to happen within one second. CI took 1.127 seconds, so setup and worker scheduling could fail the test even when the rate limit behaved correctly.

Check the real worker's completed counts after it settles: four workflows from the initial full bucket, then two more after each refill. Configure four tokens per hour for this test and backdate the exhausted bucket by half an hour to release two tokens without depending on wall-clock performance. Assert both recorded starts and committed row statuses at counts four, six, and eight.

Add normal and deliberately delayed setup cases. The 1.1-second delay reproduced the original failure before the fix; both cases now pass. This test-only change branches directly from main and is separate from the JSON output fix in #7428.

Validation:

  • PostgreSQL workflow suite: 1,048 passed.
  • Full unit suite on macOS: 11,252 passed, 160 skipped; 78.99% coverage.
  • Ruff check and format check, Pyright, and all applicable commit hooks passed.
  • Adversarial review found no actionable issues.

Review in cubic

@masenf
masenf requested a review from a team as a code owner October 6, 2026 00:27
@masenf masenf added the skip-changelog For doc/internal changes label Oct 6, 2026 — with ChatGPT Codex Connector

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 1 file

Re-trigger cubic

@greptile-apps

greptile-apps Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[Low risk] Test refactoring to remove timing flakes.

The PR appears safe to merge; no actionable issue was identified.

Summary

The test replaces wall-clock-sensitive burst assertions with checks of completed starts and committed statuses after each controlled refill. It also runs a delayed-setup case.

Reviews (1) · Last reviewed commit: "test(workflow): make rate-limit burst an..."

@codspeed

codspeed Bot commented Oct 6, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 150 untouched benchmarks
⏩ 18 skipped benchmarks1


Comparing codex/workflow-rate-test-flake (3c773bc) with main (e080fbf)

Open in CodSpeed

Footnotes

  1. 18 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@masenf
masenf merged commit d0e4b34 into main Oct 9, 2026
145 of 148 checks passed
@masenf
masenf deleted the codex/workflow-rate-test-flake branch October 9, 2026 17:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-changelog For doc/internal changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants