Skip to content

Add deterministic Node Redis queue-race eval case - #286

Open
vishaltandale00 wants to merge 1 commit into
mainfrom
codex/node-redis-command-queue-race-278
Open

Add deterministic Node Redis queue-race eval case#286
vishaltandale00 wants to merge 1 commit into
mainfrom
codex/node-redis-command-queue-race-278

Conversation

@vishaltandale00

Copy link
Copy Markdown
Owner

Summary

  • add the scoped Node Redis disconnect/command-queue synchronization case from Build the first versioned 10-case harness capability eval suite #278
  • freeze redis/node-redis v3.1.2 at commit 4f85030e42da2eed6a178e54994330af5062761e / tree 3a360d5440b2d73831123df48e24b3422676bb16 with exact Darwin arm64, Node 22.23.2, npm 10.9.8, and integrity-checked production dependencies
  • replace obsolete runtime timing with evaluator-owned synchronous socket fault injection over a real loopback RESP server
  • integrate the immutable case snapshot, materializer, production grader, mandatory gates, Eval catalog/service dispatch, and deterministic tests
  • keep the proposed upstream PR #1603 patch sealed as one admission solution; it is still open/unreleased and is not prescribed to candidates
  • do not add the shared suite manifest or any other Build the first versioned 10-case harness capability eval suite #278 case

Part of #278.

Verifier admission

npm run eval:admit:node-redis passes through the production grader.

  • case snapshot: sha256:687161980572a3f6103471b06fe5a25641ce5e83c92b7f6ecc1f7fa1ebd86b09
  • verifier: sha256:485a3139da3e8af40ec5ca8ec523be8dd5d4d3dddf6f077211aed091fd1ac953
  • untouched source baseline: red (13 failed checks)
  • upstream proposal: green
  • queue-admission alternative: green
  • clear-on-ready mutant: red (6)
  • first-error-only mutant: red (5)
  • no-reconnect mutant: red (13)
  • no-op regression mutant: red (1)
  • forged-observations/prototype-poisoning mutant: red (12)

The candidate driver signs prototype-insensitive V8-serialized observations with an ephemeral Ed25519 key generated before candidate load. A separate evaluator process verifies those bytes and independently checks the full successful server transcript. The grader also requires the committed candidate regression to be red on the untouched source and green on the candidate.

Durable local adversarial assertions record reviewer, reviewed workspace digest, verdict, and unresolved findings. Both verdicts were admissible with zero unresolved findings. They remain non-certifying local review assertions pending PR review.

Verification

Passed on head de6ee5c1116e4f7eca14c4c998a11c229228cb88:

  • npm run build
  • npx vitest run packages/eval-runner/test/node-redis-project-case.test.ts test/eval-service-simulated-user.test.mjs test/eval-app-integration.test.mjs — 3 files, 20 tests
  • npm run eval:admit:node-redis
  • git diff --check
  • individually reran the repository-wide Rust failures observed under aggregate load; all passed alone

Repository-wide unknowns

npm run check was run repeatedly but did not produce one aggregate green receipt on this host:

  • host-sensitive relayer-app-server environment tests intermittently timed out or reported Unavailable under aggregate load; every named failure passed when rerun alone
  • one run reached full Vitest and hit an unrelated macOS code-signing internal error in evidence-capture-integrity plus a too-small provider-video frame
  • one retry hit ENOSPC; only this worktree's generated Cargo cache and evaluator-created temporary Redis workspaces were cleared, after which 4.0 GiB was available
  • admission fixtures were initially collected by root Vitest; they were renamed to *.fixture.cjs, and focused/integration plus admission were rerun green afterward

Accordingly, the scoped Node Redis evidence is green, while the repository-wide aggregate check remains an explicit host/flakiness unknown rather than a claimed pass.

Upstream evidence

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant