Repository navigation
feat(adapter): share governed adapter execution with Evidence and close the delivery gaps - #2543
Conversation
…se the delivery gaps - Split the governed execution engine into registry-adapter-engine so Evidence and the facade share it. - Bound acquisition and disclosure, and document Evidence consumer guidance. - Hold async write custody and resume bounded facade writes. - Give the job host store init, scheduling, recovery and hold guidance. - Add RabbitMQ, SFTP, SQLite extract and hook sources, plus direct BReg and Coordinator destinations. - Fix fixture and example pins; express main's #2533 and #2538 fixes in the engine crate. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
…ter delivery Applies the independent review fixes on top of the delivery, resolved against main: job and pack refusals name the member and the fix, sources check their secret references and topology at load, the job host's profile quantities are bounded types, pin mismatches name the computed digest, and the generated schemas and catalogs are regenerated. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
…urce test and example The BReg Evidence source export now writes its pack under adapter-packs/<sourceId>/ with JSON schemas, and route inputs carry readable <selector>--<field> names. The prepare-source test still looked up schemas/registry-facts.yaml, and the committed breg/evidence-source-export example still carried the hashed-input digests. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
The Evidence source export now writes the registry lookup as an Adapter pack, so the two Docker lifecycle tests read the route call, constant headers and response schema from adapter-packs/<id>/pack.yaml and schemas/response.json, and the unresolved problem from protocols/registry-lookup.json, in place of the source's request block. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
…nt test The adult-status fixture moved to block YAML and an Adapter pack source connection, so the deployment test's profile rewrites, package limits and runtime document now match it. The SQLite time-budget test accepts either deadline report, since the worker and the caller's wait share one deadline and both map to the same public budget code. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
The harness picked a port by binding port 0 and closing it, then counted any answer on that port as readiness. Under parallel load another process could take the port first: the adapter then failed to bind while the harness talked to the stranger, and the next request was reset. The adapter now binds port 0 and the harness reads the bound port from its `listening` line, draining stderr on a thread so the pipe never fills. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
… in the changelog The Adapter engine move left the secret-scan allowlist for the mTLS test key on the old path, and the SFTP source tests add throwaway OpenSSH keys with no entry. Each key is pinned by its embedded public key and scoped to its test file. The docs style check wants the BReg short form introduced before the changelog uses it. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 68a3800f60
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…ce result The Adapter PostgreSQL and recovery suites run as their own path-gated CI job, and the aggregate Rust workspace result now waits for it. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
…grant journey The adult-status fixture moved to block YAML and an Adapter pack source connection, so the journey's profile rewrites, runtime source connection and package limits now match it. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
…ores A redelivery reads the event time back from its timestamptz row, which keeps microseconds, while the first hand-off carried the clock reading at nanosecond resolution. On Linux the two differed, so a destination saw a different event time on redelivery. Truncate the staged time to microseconds before it is written or handed off, so every hand-off of a sequence carries one instant. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Two fixture suites required CARGO_TARGET_DIR, which CI does not set. Use it when present and the system temp directory otherwise, as the statement-source suite already does. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Source connection endpoints are an id-unique list keyed by endpoint name, so the bundle and runtime schemas no longer declare them a set, and the runtime endpoint response bound matches the shared binding's 16 MiB ceiling. The conformance corpus records the Adapter job's product-coded integer bounds as an expected failure and the bundle's shifted positions. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
…chemas Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
…ing it unavailable Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
…er workspace Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
…nges A pending item keeps its sequence across a redeploy, but its BReg command key and Coordinator event id also hashed the job version. A retry after a version bump then looked like a new command, so BReg could execute an Action twice. Derive both from the job id, item key, sequence and digest only. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
With job-file and job-rabbitmq in one build, every RabbitMQ poll fell into the file decoder's refusal arm and stopped with adapter.job.read-bound before reading the broker. The RabbitMQ pass suite now runs with job-file enabled so that combination stays covered. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
The engine pages with OFFSET and no LIMIT, so an unbounded execution kept the server producing the rest of the result after the page was full, holding the connection until the statement timeout. The read now binds a portal and fetches at most one row past the page bound. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 92e19fbcc7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Without CARGO_TARGET_DIR the PostgreSQL runner exported the interpreter as target/breg-test-python/bin/python, relative to the repository root, while cargo runs the proof from the crate directory, so spawning it failed with NotFound. The runner now anchors a relative environment path to its own working directory. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
explain now names the pack, route, posture and typed facts instead of the retired http-json request, Functions and limits; check names deployment inputs it could not use; and the packaged example carries fewer files without per-source extract Functions. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
A fixture case now intercepts the pack endpoint and supplies the exchange it answers with; the tutorial still quoted the retired bare source response. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
…e waiter The lease race test reads pg_stat_activity inside the transaction that holds the row lock. PostgreSQL keeps the first read's snapshot for the whole transaction, so a status GET served on a connection opened after that read never appeared and the wait timed out. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Terminating every PostgreSQL session of the owning replica can fail each pooled worker loop once more before the pool notices, including the timer worker after the owner resumes, and the review worker. The journey still requires the timer failure it provokes and proves recovery by the peer. The harness now reports each cleanup failure instead of only counting them, so a log-code mismatch names the codes it saw. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
…air tests On Linux a concurrent test's fork can inherit the writer descriptor of a stub written moments ago, so exec of it fails with ETXTBSY and caseworkctl reports io-failure instead of the peer's exit class. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
…tests The PATH-fallback test execs a freshly copied evidencectl; a concurrent fork holding the copy's writer descriptor makes the first exec fail on Linux. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
… finds the address taken A pre-picked port is released before the runtime binds it, so a parallel test can take it first and the start fails with AddrInUse. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
The server stamps Date from the wall clock, so two reads that straddle a second differ there and nowhere else. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
…of closing a pipe A concurrently forked child keeps a dropped pipe's reader alive until it execs, so the audit write could still succeed and the render return 200. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
An unbounded docker exec once hung for 44 minutes on a healthy broker and stalled the whole job suite. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Shares one governed Adapter execution engine between the Adapter facade, the ingestion job host and Registry Evidence, and closes the remaining Adapter delivery gaps before v0.45.0.
registry-adapter-engineholds route compilation, execution, components and replay for the facade,adapter-joband Evidence.pack,route,connection,inputs); the taggedhttp-json/sqlite-extractform is retired and refused with a migration pointer. The upgrade rehearsal skips the Evidence leg across this boundary and says why.Twenty-nine commits, all signed off: the delivery, the review fixes, three fixes from automated review, three fixes from merge-queue runs, eight test-stability fixes, and thirteen follow-ups that keep the rest of the repository in step. Five are test-only and follow the new Adapter pack layout and fixture format (the bregctl prepare-source test and committed example, the Docker-only BReg dev lifecycle tests, the Evidence client deployment test, the Casework Evidence task-grant journey) or take the served Adapter's port from its own listening line instead of a pre-picked port another process could take. One points the secret-scan allowlist at the moved mTLS test key and the new SFTP test keys and introduces the BReg short form in the docs changelog. One teaches the release-tooling test that the Rust workspace result now waits for the Adapter PostgreSQL lane. One stores the staged item time at the microsecond precision PostgreSQL keeps, so a Coordinator event redelivered from the stored row carries the same event time as the first hand-off. One lets the Evidence fixture tests fall back to the system temporary directory when no Cargo target directory is set. One aligns the Evidence bundle and runtime connection schemas with the runtime (endpoint bindings are identified lists checked by name, and the runtime response-size bound matches the code) and records the Adapter job's product-coded quantity bounds in the configuration conformance expected failures. One regenerates the identifier catalog for those schema digests. One gives the AMQPS fixture broker up to a minute to boot before the private-CA test calls it unavailable. One raises the identifier catalog job's time limit from 10 to 20 minutes, since its cold build already took over nine minutes on main and the larger workspace pushed it past ten.
The three fixes from automated review:
rabbitmq-jsonprofile is decoded by the broker path. The RabbitMQ lane of the job PostgreSQL gate now also carries thejob-filefeature to keep that build proven.The three fixes from merge-queue runs:
CARGO_TARGET_DIRwas unset. The path is now absolute, with a runner test for the unset case.explainoutput (pack, route, posture, typed facts), thecheckline that names deployment inputs it could not use, and the smaller packaged example.The eight test-stability fixes are test-only and address timing failures seen in merge-queue runs:
pg_stat_activityread for the whole transaction and a status GET on a newer connection stayed invisible.ETXTBSYon a freshly written executable, with a bounded retry that fires on that error only.Dateheader.dockerandrabbitmqctlcalls.Local checks on the same tree:
cargo fmt --check,cargo clippy --workspace --all-targets -D warningsandcargo deny checkclean; Adapter facade PostgreSQL suite ok; Adapter job PostgreSQL suite 190 passed, 0 failed; release tooling tests 51 ok; BReg PostgreSQL runner tests 16 ok; Evidence tutorial dry run ok; configuration conformance (strict) clean for the Adapter and Evidence formats; Evidencecheck-contracts.shok; configuration convention lint ok. A known Render serving test flake is unrelated to this change.