Repository navigation
Add Codex as an advisory red-team audit engine - #1263
Conversation
`scripts/security-audit-local.sh --engine codex` runs the same audit prompts on gpt-6-sol under the daybreak_blue Cyber access program, sandboxed to the working tree. Its canary takes a CI canary run's seed key (`--key <run ID>`) and that run's decrypted scorecard to count the seeds either engine caught, then prints the publishing command. `.github/workflows/codex-audit-report.yaml` publishes from non-secret dispatch inputs only (commit, model, verdicts, canary counts). The public-body builder checks each input, the commit against main, the dispatcher's admin role, and the counts against the CI canary run's own record, then comments on the status issue and uploads `codex-recall`. The recall graph on /security draws three lines: Claude, Codex, and caught by either. security-audit.md keeps S8's Claude split as the release gate and adds a "Red team" section with its cadence (quarterly, plus before releases that change remote control or Hosted) and S275. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Deploying mouseterm with
|
| Latest commit: |
60c8467
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://bbfd2dc2.mouseterm.pages.dev |
| Branch Preview URL: | https://codex-advisory-engine.mouseterm.pages.dev |
dormouse-bot
left a comment
There was a problem hiding this comment.
The local Codex canary seeds the checkout's HEAD, not the commit of the run its --key names. In run_canary, git archive HEAD | tar -x -C "$tree" builds the tree whatever --key says. The Red team section says to key it by "the latest CI canary run on main". That run is weekly and main usually moves past it, so a maintainer who runs from an up-to-date main gets the following:
unionpasses, because both scorecards hold the same seed IDs from the same pool.- The script prints a dispatch command with
commit=<local HEAD>. checkAgainstClaudethen rejects it ("seeded X, not Y"), after a full five-domain Codex canary has already run.
Publishing stays correct; the problem is that the quarterly run is wasted, and the requirement to be at that commit appears only in a spec sentence ("when run at its commit"). One fix makes this correct by construction: when --key is a run ID, read its head_sha (gh api repos/$GITHUB_REPOSITORY/actions/runs/$key --jq .head_sha), git archive that commit, and pass it to print_codex_report as the commit. That would also drop the uncommitted-changes warning print_codex_report prints on this path, which doesn't apply to an archived tree.
listCodexRuns's second admin check has no trigger (generate-canary-recall.js). The query already keeps only status=success runs, and publishCodex throws for any non-admin GITHUB_TRIGGERING_ACTOR. So a successful run exists only if an admin triggered it, re-runs included. The only case the re-check changes is one where nothing was bypassed: an admin who is later demoted loses their records from the published graph. It also costs an API call per actor and a spec clause ("counts only from an admin's dispatch"). Dropping the role filter, and that clause, keeps the same guarantee with less code.
`--key <run ID>` now reads that run's `head_sha` and archives it, so a maintainer on an up-to-date `main` seeds what the CI run seeded and the printed dispatch names the commit `codex-audit-report.yaml` will accept. The recall generator drops its second admin check: only an admin's dispatch of the report workflow can succeed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Both addressed in 60c8467:
|
|
Both look resolved in 60c8467. With One small asymmetry, and it fails safe: |
Summary
scripts/security-audit-local.sh --engine codexruns the same.github/audit/prompts ongpt-6-solwith--cyber-access-program daybreak_blue, sandboxed (workspace-write+ network).canary --key <run ID>seeds what that CI canary run seeded;--claude-scorecard(the run's decryptedscorecard.json) adds the "caught by either" count. It prints thegh workflow run codex-audit-report.yaml …command and files nothing..github/workflows/codex-audit-report.yaml:workflow_dispatchwith only non-secret inputs (commit, model, verdicts, canary counts).security-audit-public-body.mjs --engine codexvalidates every input, requires the commit onmainand an admin dispatcher, and checks the counts against the CI canary run's owncanary-recallrecord and commit. It then comments once on the status issue and uploads acodex-recallartifact. Onlyissues: writeon its one job (S45 exception added tosecurity-ci.mdandworkflow-lint.mjs).codex-recallrecord with its canary run and fails the snapshot on an impossible pairing.security-audit.md: S8's Claude split stays the release gate. A new Red team section sets the cadence (quarterly, plus before releases that change remote control or Hosted) and addsS275. The embargo table lists the new public sinks, and rationale entries are added.Test plan
pnpm test:checks(spec, md, public-docs, workflow lints and their tests)pnpm test:audit: 180 pass, including new S275 workflow/builder tests and the local Codex engine and canary-union testswebsitepnpm test: typecheck + 456 vitest/securitywith a patched snapshot to check the three-line graph--engine codexrun: not exercised here🤖 Generated with Claude Code