raw.json

{"summary":"Strong fail-closed scanner, artifact-reader, and verification contracts are undermined by two unsafe GitHub artifact-download paths, contradictory suppression semantics, and several public-surface inconsistencies.","scores":{"semantic_correctness":88,"false_positive_control":83,"security_trust_boundaries":68,"verification_design":92,"scope_contract_consistency":82,"docs_usability":84},"findings":[{"severity":"H","category":"security_trust_boundaries","file":"skills/cypress-debugger/SKILL.md","line":119,"title":"Cypress download bypasses artifact safety boundary","evidence":"The documented command gives gh direct extraction control over cypress/reports, despite this file requiring no-follow destination validation and revalidation before artifact writes. Unlike the Playwright path, it provides no bounded, validated, atomic extraction helper for the untrusted archive.","recommendation":"Add a Cypress artifact-download helper with fixed artifact selection, fork-run rejection, bounded ZIP validation, descriptor-relative extraction, and atomic no-replace publication; replace this command with that helper."},{"severity":"H","category":"security_trust_boundaries","file":"skills/playwright-debugger/scripts/download-playwright-report.py","line":181,"title":"Artifact helper executes ambient gh binary","evidence":"The helper invokes the bare executable name gh and subprocess execution inherits the caller environment. A PATH-selected executable therefore sits inside the credentialed download boundary, unlike the packet's explicit binding of curl and scanner tools to trusted absolute paths.","recommendation":"Resolve and validate an absolute gh executable before entering the workspace, execute it with a minimal allowlisted environment, and reject project-controlled or writable executable paths."},{"severity":"M","category":"false_positive_control","file":"skills/e2e-reviewer/SKILL.md","line":152,"title":"Suppression contract contradicts scanner behavior","evidence":"This line says a JUSTIFIED hit must be skipped, while the later scanner contract retains mechanically suppressed P0 hits as JUSTIFIED-REVIEW candidates and p0-candidate can still fail on them. Users cannot predict whether a documented suppression removes or preserves a candidate.","recommendation":"State uniformly that justified P1/P2 hits are skipped, justified P0 hits remain non-final review candidates, and focused-test leaks are never suppressible."},{"severity":"M","category":"scope_contract_consistency","file":"README.md","line":164,"title":"Inline fallback overstates verdict equivalence","evidence":"The README broadly promises that the skills preserve the same inline verdict when delegation is unavailable, but the generator's V6 contract explicitly forbids inline self-review from producing PASS and requires CANNOT_VERIFY when separation is unavailable.","recommendation":"Limit the inline-equivalence statement to reviewer finding verification and debugger classification, and document that generator V6 becomes CANNOT_VERIFY/PARTIAL without an independent context."},{"severity":"M","category":"docs_usability","file":"README.md","line":154,"title":"Codex-specific install also targets Claude Code","evidence":"The recommended command in the Codex section includes both -a claude-code and -a codex, so following Codex-only guidance can modify an additional host installation without the surrounding text disclosing that effect.","recommendation":"Use only -a codex in the Codex command, or explicitly label the existing command as a dual-host installation and provide a Codex-only variant."}],"limitations":["No commands, tests, scanners, benchmarks, holdouts, network checks, or runtime probes were run.","The packet deliberately omits benchmark results, prior reviews, repository history, and many CI helper implementations; no outcome or evidence-integrity claims were inferred from them.","Scores assess only included contracts and implementation excerpts, not observed detection accuracy or runtime compatibility."],"verdict":"FAIL"}