feat(core): append-only detector_rca_executions with pre-run allocation and CAS latest pointer - #2070
Conversation
PR overviewThis pull request adds append-only RCA execution records, deterministic trace metadata, concurrency-safe result updates, and execution details to the finding RCA API. 3 security concerns remain open, the most serious being [GuardDog] Suspicious package: @prisma/client@5.22.0 (threat-process-hooks, capability-process-hooks, capability-process-spawn). Open issues (3)
Scanned with GuardDog · Semgrep · TruffleHog · Trident review. OSV failed to run. View in Trident Fixed/addressed: 3 · PR risk: 10/10 |
There was a problem hiding this comment.
1 issue found across 13 files
Confidence score: 2/5
frontend/packages/core/src/rca-executions.ts: The RCA worker does not invoke the new helpers, so RCA jobs create nodetector_rca_executionsrows or advancelatestExecutionId; trace, attempt, and link surfaces remain null. Wire allocation beforerunRcaSes...and verify the worker persists and advances execution state.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="frontend/packages/core/src/rca-executions.ts">
<violation number="1" location="frontend/packages/core/src/rca-executions.ts:17">
P1: The RCA worker never invokes these helpers, so RCA jobs create no `detector_rca_executions` rows or advance `latestExecutionId`; the new trace, attempt, and link surfaces remain null. Wire allocation before `runRcaSession`, then update execution status and the latest pointer on completion.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Fix all with cubic | Re-trigger cubic
| * transaction holding the finding's DetectorRca row lock, so two concurrent | ||
| * allocations get attempts n and n+1 — never the same trace id. | ||
| */ | ||
| export async function allocateExecution( |
There was a problem hiding this comment.
P1: The RCA worker never invokes these helpers, so RCA jobs create no detector_rca_executions rows or advance latestExecutionId; the new trace, attempt, and link surfaces remain null. Wire allocation before runRcaSession, then update execution status and the latest pointer on completion.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At frontend/packages/core/src/rca-executions.ts, line 17:
<comment>The RCA worker never invokes these helpers, so RCA jobs create no `detector_rca_executions` rows or advance `latestExecutionId`; the new trace, attempt, and link surfaces remain null. Wire allocation before `runRcaSession`, then update execution status and the latest pointer on completion.</comment>
<file context>
@@ -0,0 +1,107 @@
+ * transaction holding the finding's DetectorRca row lock, so two concurrent
+ * allocations get attempts n and n+1 — never the same trace id.
+ */
+export async function allocateExecution(
+ db: Db,
+ params: { findingId: string; projectId: string },
</file context>
There was a problem hiding this comment.
Expected here — this PR adds the executions table and its helpers; the RCA worker is wired to them in #2072 (detector-rca-processor.ts calls allocateExecution before the run and advanceLatest after). The stack is split so schema and call sites review separately; nothing in this PR is reachable until that one lands, which is also why the trace/attempt surfaces read null throughout it.
e678468 to
d545418
Compare
cce77ef to
123dcf8
Compare
6262129 to
69e6232
Compare
There was a problem hiding this comment.
All reported issues were addressed across 12 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Fix all with cubic | Re-trigger cubic
…on and CAS latest pointer Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018DopjNH21gLaDtWeeMDEHX
…CA reads Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018DopjNH21gLaDtWeeMDEHX
…n session Two review findings, both real. **Node-only module on the client barrel.** rca-executions imports node:crypto and prisma, and `export * from "./rca-executions.ts"` in index.ts put it on the path every client component takes when it imports "@traceroot/core" — the exact thing the comment above that line says pi-ai is excluded to avoid. It ships as a subpath export instead. A later PR in this stack already removed the line; doing it here means this PR is correct on its own rather than transiently broken. A test pins the convention so it cannot drift back. **Session fell back across attempts.** `row.latestExecution?.sessionId ?? row.sessionId` reaches for the legacy column whenever the latest execution's session is null — which is not "no session recorded" but "this run has no chat yet", so the UI could open the *previous* attempt's conversation. The legacy column is now consulted only when there is no execution row at all, matching traceId and traceStatus beside it, which already never fall back. Co-Authored-By: Claude <noreply@anthropic.com>
…s on max attempt "The current execution of a finding" is now simply the row with the highest attempt. That removes latest_execution_id (column, @unique, relation, back- relation) and the read-then-write CAS in advanceLatest, whose pointer only moved on success and so left "latest" ambiguous between "latest success" and "highest attempt" — the ambiguity that let an older attempt strand a finding at status=running. finishFindingIfLatest replaces it: one UPDATE detector_rcas ... WHERE NOT EXISTS (a higher attempt), used by both the success and failure paths, so the database — not a comment — enforces that only the current attempt speaks for the finding, with no read-then-write window. Also: - delete setExecutionTraceStatus (thin wrapper; callers do a single update) - delete the legacy lazy-seed block: nothing reads the synthetic attempt-1 row it wrote, and the attempt-1 = dashless-finding-id convention holds for legacy findings without it - drop the write-only result column and the unused project_id index from detector_rca_executions - add onUpdate: NoAction to both new relations so `prisma migrate diff --to-schema-datamodel` matches the migration SQL (repo convention) Tests now use a fake backed by a real executions list: aggregate reflects prior creates, create throws on a duplicate (finding, attempt), and $executeRaw evaluates the NOT EXISTS predicate — so the uniqueness and "no higher attempt" properties are asserted rather than stubbed away. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015DXLChvn8HPA42wsuDUFYo
The pointer is gone; the RCA route now reads `executions` ordered by attempt desc, take 1, selecting only the four fields the response uses instead of including the whole row. During a re-run this is the running attempt, so a null session there means "no chat yet", never the previous attempt's chat. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015DXLChvn8HPA42wsuDUFYo
trace_id/trace_status/attempt named an internal (source='detector') trace that the public trace endpoints refuse under customer_traffic_only(), so the public contract advertised an id it could not serve — and nothing consumed the fields. They stay on the internal Next.js RCA route only. RCAResult is the one model behind both the public and the internal backend finding routes, so _read_rca goes back to SELECT status, result with no execution join; public.json regenerated via scripts/sync_public_openapi.py. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015DXLChvn8HPA42wsuDUFYo
348db78 to
ef30639
Compare
…nding row lock The guarded UPDATE evaluated its NOT EXISTS (higher attempt) against the statement's own snapshot. A concurrent allocateExecution holding the detector_rcas row lock could commit attempt n+1 after that snapshot was taken, so a slow attempt n could still overwrite the retry's state. Run the finish in a transaction that first takes the same SELECT ... FOR UPDATE allocation takes. The guard now runs only once any in-flight allocation has committed (and is visible under READ COMMITTED's per-statement snapshot) or is blocked until we finish. The fake db in the tests hands out a distinct tx client and refuses raw SQL on the top-level client, so the test proves both statements run inside the transaction and the lock precedes the UPDATE; a new case commits attempt 2 exactly when the lock is granted and asserts the finish yields. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015DXLChvn8HPA42wsuDUFYo
…ry is pending "Current execution = highest attempt" is right for status, attempt and session, but taking the trace from it blanked out a working link the moment a retry was allocated (its trace is pending until the run ends). The RCA route now reads the finding's executions newest-first and splits the two roles: attempt/sessionId come from the current execution; traceId/ traceStatus come from the newest execution whose trace is `available`, falling back to the current one's pending/failed/disabled status only when none is. No pointer column needed — the rule is a one-line read. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015DXLChvn8HPA42wsuDUFYo
Rebased onto the fixed attribution tip. #2070 dropped advanceLatest, setExecutionTraceStatus, detector_rcas.latest_execution_id and detector_rca_executions.result; the worker now finishes a finding through finishFindingIfLatest (one guarded write for done and failed alike), writes the execution row's traceStatus/sessionId/finishedAt in one update, and gates the pre-run "running" flip on the attempt still being the highest (markFindingRunningIfLatest). This PR's own failFindingIfLatest is gone — it duplicated the new helper. runRcaSession's execution identity is required; a stream that carried output but no `trace` frame records `failed`. #2071 made attribution a required appendMessage argument and let the persister charge an external capture budget: the route now builds the persister inside the traced run and hands it the run's own accumulator, so spans and rows share one budget. The trace kind is derived from the turn's attribution, and the parent-execution lookup for a follow-up only runs when that follow-up will be traced and can no longer fail the turn. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015DXLChvn8HPA42wsuDUFYo
The runs proxy read `detector_rcas.latest_execution_id`, which #2070 dropped in favour of the append-only executions table. Read the finding's executions newest-attempt first and apply the same rule as the RCA route: the status is the current attempt's, the trace link is the newest attempt whose export landed (so a pending retry does not hide a working trace), falling back to the current attempt's pending/failed/disabled status. `TraceStatus` is now exported from @traceroot/core (#2072); the ai-assistant types import it from there instead of reaching into the detectors hook module. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015DXLChvn8HPA42wsuDUFYo
Rebased onto the fixed attribution tip. #2070 dropped advanceLatest, setExecutionTraceStatus, detector_rcas.latest_execution_id and detector_rca_executions.result; the worker now finishes a finding through finishFindingIfLatest (one guarded write for done and failed alike), writes the execution row's traceStatus/sessionId/finishedAt in one update, and gates the pre-run "running" flip on the attempt still being the highest (markFindingRunningIfLatest). This PR's own failFindingIfLatest is gone — it duplicated the new helper. runRcaSession's execution identity is required; a stream that carried output but no `trace` frame records `failed`. #2071 made attribution a required appendMessage argument and let the persister charge an external capture budget: the route now builds the persister inside the traced run and hands it the run's own accumulator, so spans and rows share one budget. The trace kind is derived from the turn's attribution, and the parent-execution lookup for a follow-up only runs when that follow-up will be traced and can no longer fail the turn. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015DXLChvn8HPA42wsuDUFYo
The runs proxy read `detector_rcas.latest_execution_id`, which #2070 dropped in favour of the append-only executions table. Read the finding's executions newest-attempt first and apply the same rule as the RCA route: the status is the current attempt's, the trace link is the newest attempt whose export landed (so a pending retry does not hide a working trace), falling back to the current attempt's pending/failed/disabled status. `TraceStatus` is now exported from @traceroot/core (#2072); the ai-assistant types import it from there instead of reaching into the detectors hook module. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015DXLChvn8HPA42wsuDUFYo
Rebased onto the fixed attribution tip. #2070 dropped advanceLatest, setExecutionTraceStatus, detector_rcas.latest_execution_id and detector_rca_executions.result; the worker now finishes a finding through finishFindingIfLatest (one guarded write for done and failed alike), writes the execution row's traceStatus/sessionId/finishedAt in one update, and gates the pre-run "running" flip on the attempt still being the highest (markFindingRunningIfLatest). This PR's own failFindingIfLatest is gone — it duplicated the new helper. runRcaSession's execution identity is required; a stream that carried output but no `trace` frame records `failed`. #2071 made attribution a required appendMessage argument and let the persister charge an external capture budget: the route now builds the persister inside the traced run and hands it the run's own accumulator, so spans and rows share one budget. The trace kind is derived from the turn's attribution, and the parent-execution lookup for a follow-up only runs when that follow-up will be traced and can no longer fail the turn. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015DXLChvn8HPA42wsuDUFYo
The runs proxy read `detector_rcas.latest_execution_id`, which #2070 dropped in favour of the append-only executions table. Read the finding's executions newest-attempt first and apply the same rule as the RCA route: the status is the current attempt's, the trace link is the newest attempt whose export landed (so a pending retry does not hide a working trace), falling back to the current attempt's pending/failed/disabled status. `TraceStatus` is now exported from @traceroot/core (#2072); the ai-assistant types import it from there instead of reaching into the detectors hook module. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015DXLChvn8HPA42wsuDUFYo
Rebased onto the fixed attribution tip. #2070 dropped advanceLatest, setExecutionTraceStatus, detector_rcas.latest_execution_id and detector_rca_executions.result; the worker now finishes a finding through finishFindingIfLatest (one guarded write for done and failed alike), writes the execution row's traceStatus/sessionId/finishedAt in one update, and gates the pre-run "running" flip on the attempt still being the highest (markFindingRunningIfLatest). This PR's own failFindingIfLatest is gone — it duplicated the new helper. runRcaSession's execution identity is required; a stream that carried output but no `trace` frame records `failed`. #2071 made attribution a required appendMessage argument and let the persister charge an external capture budget: the route now builds the persister inside the traced run and hands it the run's own accumulator, so spans and rows share one budget. The trace kind is derived from the turn's attribution, and the parent-execution lookup for a follow-up only runs when that follow-up will be traced and can no longer fail the turn. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015DXLChvn8HPA42wsuDUFYo
The runs proxy read `detector_rcas.latest_execution_id`, which #2070 dropped in favour of the append-only executions table. Read the finding's executions newest-attempt first and apply the same rule as the RCA route: the status is the current attempt's, the trace link is the newest attempt whose export landed (so a pending retry does not hide a working trace), falling back to the current attempt's pending/failed/disabled status. `TraceStatus` is now exported from @traceroot/core (#2072); the ai-assistant types import it from there instead of reaching into the detectors hook module. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015DXLChvn8HPA42wsuDUFYo
The runs proxy read `detector_rcas.latest_execution_id`, which #2070 dropped in favour of the append-only executions table. Read the finding's executions newest-attempt first and apply the same rule as the RCA route: the status is the current attempt's, the trace link is the newest attempt whose export landed (so a pending retry does not hide a working trace), falling back to the current attempt's pending/failed/disabled status. `TraceStatus` is now exported from @traceroot/core (#2072); the ai-assistant types import it from there instead of reaching into the detectors hook module. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015DXLChvn8HPA42wsuDUFYo
Rebased onto the fixed attribution tip. #2070 dropped advanceLatest, setExecutionTraceStatus, detector_rcas.latest_execution_id and detector_rca_executions.result; the worker now finishes a finding through finishFindingIfLatest (one guarded write for done and failed alike), writes the execution row's traceStatus/sessionId/finishedAt in one update, and gates the pre-run "running" flip on the attempt still being the highest (markFindingRunningIfLatest). This PR's own failFindingIfLatest is gone — it duplicated the new helper. runRcaSession's execution identity is required; a stream that carried output but no `trace` frame records `failed`. #2071 made attribution a required appendMessage argument and let the persister charge an external capture budget: the route now builds the persister inside the traced run and hands it the run's own accumulator, so spans and rows share one budget. The trace kind is derived from the turn's attribution, and the parent-execution lookup for a follow-up only runs when that follow-up will be traced and can no longer fail the turn. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015DXLChvn8HPA42wsuDUFYo
The runs proxy read `detector_rcas.latest_execution_id`, which #2070 dropped in favour of the append-only executions table. Read the finding's executions newest-attempt first and apply the same rule as the RCA route: the status is the current attempt's, the trace link is the newest attempt whose export landed (so a pending retry does not hide a working trace), falling back to the current attempt's pending/failed/disabled status. `TraceStatus` is now exported from @traceroot/core (#2072); the ai-assistant types import it from there instead of reaching into the detectors hook module. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015DXLChvn8HPA42wsuDUFYo
Rebased onto the fixed attribution tip. #2070 dropped advanceLatest, setExecutionTraceStatus, detector_rcas.latest_execution_id and detector_rca_executions.result; the worker now finishes a finding through finishFindingIfLatest (one guarded write for done and failed alike), writes the execution row's traceStatus/sessionId/finishedAt in one update, and gates the pre-run "running" flip on the attempt still being the highest (markFindingRunningIfLatest). This PR's own failFindingIfLatest is gone — it duplicated the new helper. runRcaSession's execution identity is required; a stream that carried output but no `trace` frame records `failed`. #2071 made attribution a required appendMessage argument and let the persister charge an external capture budget: the route now builds the persister inside the traced run and hands it the run's own accumulator, so spans and rows share one budget. The trace kind is derived from the turn's attribution, and the parent-execution lookup for a follow-up only runs when that follow-up will be traced and can no longer fail the turn. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015DXLChvn8HPA42wsuDUFYo
The runs proxy read `detector_rcas.latest_execution_id`, which #2070 dropped in favour of the append-only executions table. Read the finding's executions newest-attempt first and apply the same rule as the RCA route: the status is the current attempt's, the trace link is the newest attempt whose export landed (so a pending retry does not hide a working trace), falling back to the current attempt's pending/failed/disabled status. `TraceStatus` is now exported from @traceroot/core (#2072); the ai-assistant types import it from there instead of reaching into the detectors hook module. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015DXLChvn8HPA42wsuDUFYo
detector_rcas.finding_idis UNIQUE, so a retry overwrites the previous RCA and no record survives that an earlier attempt happened — and there is nowhere to put a trace id.Adds append-only
detector_rca_executions(attempt,trace_id,trace_status,session_id,started_at,finished_at):attemptandtrace_id— is created under the finding's row lock before the agent starts (allocateExecution). Writing ids after the run is not crash-safe: a crash between export and DB write would let the retry reuse the first trace id and produce a second root span in one trace. Attempt 1's trace id is the finding id (dashless); later attempts hashfinding:attempt.trace_statusis a 4-state enum (disabled/pending/available/failed), not a boolean, so a link is never offered for a trace that was never recorded.finishFindingIfLatest, which runs in a transaction that takes thedetector_rcasrowFOR UPDATEand updates only when no higher attempt exists — a slow older run finishing last cannot overwrite a newer one.markFindingRunningIfLatestguards the pre-runrunningwrite the same way.available(so a retry in flight does not hide the previous, openable trace). The public findings API deliberately exposes onlyrca.status/rca.result: the agent trace is not readable through the public trace endpoints, so advertising its id there would be misleading.onUpdate: NoActionon both sides so the migration and the Prisma datamodel agree (no permanent drift underprisma migrate dev).No backfill. Pre-existing RCAs have no execution row; every surface reads that as "no link". A legacy finding's first traced re-run is attempt 1, deliberately: it never had a traced execution.
Closes #2060. Part of #2058.
Stack (agent self-trace, #2058)
Merge in this order; each PR's base branch is the one above it. Reviewed end to end (Fable 5.1 audit + codex cross-review); all nine are CI-green on their current heads.
@traceroot-ai/traceroot0.4.0 reaching npm (Publish 0.4.0: instrumentPiAgentCore for @earendil-works/pi-agent-core traceroot-ts#150); thefile:tarball pin must be swapped before this and everything above it mergeIndependent: #2075 (digest self-trace) is based on
mainand can land at any time; it and #2071 both touchfrontend/worker/src/detection/, so whichever lands second rebases.Integration branch with all of the above plus the handoff note, the seeded-data fixtures and the e2e journey test:
feat/agent-trace-all.Co-Authored-By: Claude noreply@anthropic.com
https://claude.ai/code/session_015DXLChvn8HPA42wsuDUFYo