Stack 3/4: expose Ladybug graph search to Codex - #320
Conversation
c5d01f3 to
567ccbf
Compare
750b5b4 to
02791a0
Compare
vishaltandale00
changed the title
Apple-Silicon graph-search handoffThe authorized Apple-Silicon delivery slice for #52 is implemented as a three-PR stack:
Exact final head: Evidence ledger
Adversarial findingsThe review first found that reference authorship could precede the alleged search and that no concurrent same-anchor draft challenged accepted-only isolation; both were fixed. The first live run then exposed that a successful turn-1 search could be selected as turn-2 evidence; the evaluator now records an explicit second-turn boundary and rejects all earlier searches. The hardened deterministic and live runs pass. Final certifying review covered diff digest RemainingThe PRs remain unmerged. Merge in stack order (#313, #319, #320), preserving exact heads or rerunning invalidated evidence after rebases. GitHub issues remain open until their corresponding stack layers land. Windows and Intel macOS support remain deferred rather than silently claimed. |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5d29fe1aa2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (method === "POST" && path === "/api/graph/actions") return withRecord(event, "action", response?.action); | ||
| if (method === "POST" && path === "/api/graph/actions") return withAction(event, response?.action); | ||
| if (method === "POST" && /^\/api\/graph\/layers\/\d+\/discard$/.test(path)) return withRecord(event, "layer", response?.layer); | ||
| if (method === "POST" && path === "/api/graph/search" && status >= 200 && status < 300) return withSearchResult(event, response); |
There was a problem hiding this comment.
When live Codex performs any successful one-result search—such as MATCH (l:Layer) RETURN l AS layer in this isolated run—and references the returned first layer, this audit records only the response layer IDs, so checkGraphMemorySecondTurn passes without proving that the required anchor query and tagged parameter were used. This can falsely certify the paid graph-memory evaluation despite the PRD requiring the exact bounded search and prohibiting a hard-coded or inferred target; sanitize and checkpoint the search request's version, query, and anchor parameter as well as its response.
AGENTS.md reference: AGENTS.md:L18-L20
Useful? React with 👍 / 👎.
# Conflicts: # packages/harness-host/test/configuration.test.ts
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: da610e3acf
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| The current interaction may carry an invoke lease created by the product. Before authoring, use graph.getNode(${interactionNode.id}) and graph.getNeighbors(${interactionNode.id}) to inspect the current node and any relevant source context exposed by the graph. Treat that context as input to your answer; do not copy, forge, or manage lease metadata. Author the response normally. A successful ordinary graph.submit(${interactionNode.id}) automatically fulfills any lease held by this interaction. There is no separate resolveAction call. | ||
|
|
||
| Graph search is available through the same executable JavaScript client as await graph.search(request, options). It is not a provider-native tool or MCP function. The public search request is capability-scoped to the current interaction's thread and accepts exactly queryContractVersion, query, optional parameters, and optional budget. Never add target, thread, project, scope, permit, credential, database, candidate-source, or other authority fields. Search sees accepted published graph records only; it never exposes drafts and never falls back to SQLite when the Ladybug index is unavailable. |
There was a problem hiding this comment.
This rebased change materially alters the codex.basic prompt, but harnesses/codex-basic.yaml remains byte-identical to the parent at revision 4 because an intervening main commit had already consumed the revision that the original graph-search stack used. Consequently, executions with and without this search guidance can record the same configuration revision and harnessConfigurationDigest, undermining the PRD-required exact snapshot/digest identity used for Eval and promotion comparisons; advance the affected configuration to a new revision.
AGENTS.md reference: AGENTS.md:L18-L18
Useful? React with 👍 / 👎.
| ...base, | ||
| { | ||
| name: "search-returned-prior-root", | ||
| passed: successfulSearches.length === 1 |
There was a problem hiding this comment.
When a later action or submission fails validation, the harness guidance tells Codex to repair and rerun the same whole authoring program, which legitimately repeats the successful anchor search before creating the final matching reference. Requiring exactly one successful search therefore marks an otherwise correct and fully ordered live run as failed; the PRD requires the exact prior layer to be searched before the reference, not that the search occur only once, so reconcile repeated qualifying searches instead of failing solely on their count.
AGENTS.md reference: AGENTS.md:L18-L18
Useful? React with 👍 / 👎.
|
|
||
| The current interaction may carry an invoke lease created by the product. Before authoring, use graph.getNode(${interactionNode.id}) and graph.getNeighbors(${interactionNode.id}) to inspect the current node and any relevant source context exposed by the graph. Treat that context as input to your answer; do not copy, forge, or manage lease metadata. Author the response normally. A successful ordinary graph.submit(${interactionNode.id}) automatically fulfills any lease held by this interaction. There is no separate resolveAction call. | ||
|
|
||
| Graph search is available through the same executable JavaScript client as await graph.search(request, options). It is not a provider-native tool or MCP function. The public search request is capability-scoped to the current interaction's thread and accepts exactly queryContractVersion, query, optional parameters, and optional budget. Never add target, thread, project, scope, permit, credential, database, candidate-source, or other authority fields. Search sees accepted published graph records only; it never exposes drafts and never falls back to SQLite when the Ladybug index is unavailable. |
There was a problem hiding this comment.
This guidance is emitted for every codex.basic turn, including macOS x64 and Windows x64 development packages, even though the PRD explicitly keeps graph search unsupported on those targets until their qualification gates pass. Those platforms remain configured desktop targets, their package builds compile the graph server with Ladybug enabled by default, and the packaged-client verifier requires the search-enabled client without a platform condition, so they expose and encourage an unqualified search path rather than withholding it; gate the prompt and runtime/package surface on the qualified platform.
AGENTS.md reference: AGENTS.md:L18-L19
Useful? React with 👍 / 👎.
vishaltandale00
changed the title
|
@codex review |
|
Codex Review: Didn't find any major issues. 🎉 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
1 participant
Final Apple-Silicon stack layer for #52 and #266, stacked on #319.
This replaces the obsolete Prime-oriented prototype with the existing packaged
codex.basicharness path. It does not merge or release anything, and it does not claim Windows or Intel macOS graph-search support.Product outcome
graph.search()through the same bundled executable TypeScript client used for graph authoring.RelayerGraphClient.prototype.searchin the desktop graph client.codex-basicconfiguration revision so product threads receive the new guidance.Deterministic product proof
Adds
graph-memory.prior-accepted-referenceandnpm run eval:graph-memory.The inference-free path starts the real Rust graph server, Ladybug index, Node harness host, and public TypeScript client. It proves:
graph.search()returns exactly the prior accepted root layer;referenceaction targets that exact searched identity;Exact retained local run:
issue-52-final-deterministic-3. Result: PASS.Live Codex product proof
Exact retained metered run at
5d29fe1aa2c8c2a1308b0ff975d74a2558acb4d4:Result: PASS. One Codex harness object handled both interactions in the same persisted Codex session with distinct, later-revoked capabilities. Submit 1 acknowledged at audit sequence 23; the second-turn search at sequence 32 returned exactly layer 1; the typed reference at sequence 35 targeted layer 1; submit 2 acknowledged at sequence 37. Harness configuration digest:
sha256:f906d7dd7dc3cd92402addca69ef3e796b38d13a2940d1ab314044ea2af8d434.The first live attempt correctly exposed an evaluator weakness: an optional successful search from turn 1 could be selected as the alleged turn-2 evidence. The product behavior itself succeeded, but the run was retained as failed. The evaluator now records an explicit second-turn boundary and rejects all earlier searches; deterministic regression and the second live run pass that hardened boundary.
Paid inference remains opt-in and is not part of the default suite.
Exact local verification
At
5d29fe1aa2c8c2a1308b0ff975d74a2558acb4d4:npm run check— PASSnpm run build— PASSnpm run eval:graph-memory -- --output-dir .relayer/evals/graph-memory-final-3 --test-run-id issue-52-final-deterministic-3— PASSgit diff --check— PASSOne earlier full-check attempt hit an ambient wall-time timeout in the unchanged Ladybug aggregate-allowlist chapter. Its exact focused rerun passed immediately, and the subsequent full
npm run checkpassed on this head.Adversarial assertion
Reviewer:
final_graph_search_adversary.The main implementation review covered exact workspace digest
4cb71d3594ee8b336576c27b25c431415e5699de, which became02791a0b281f5a78636821b5e475420ff691c5a2without source changes. A follow-up review covered exact diff digest784d9f6354f81d1ca972baf55ff991d7de2182f6, which became5d29fe1aa2c8c2a1308b0ff975d74a2558acb4d4without further changes.Scope: PRD/spec fidelity, authority scoping, public-client packaging, prompt/query correctness, accepted/draft isolation, same-session distinct-capability proof, exact search/reference identity and response ordering, no fallback, opt-in inference boundaries, and resistance to pre-turn search-evidence laundering.
Verdict: certifying PASS, no unresolved findings.
Stack and remaining gate
Hosted
checkand the macOS ARM target package passed on exact head5d29fe1ain run 33321019108. Windows is intentionally skipped and Intel macOS is out of this narrowed release slice. No merge is requested.