raw.json

{"summary":"The packet defines strong fail-closed review, verification, and artifact-reading contracts, but material implementation and guidance gaps remain: scanner inputs can change without identity detection, documented artifact-download commands bypass the required symlink-safe write gate, and URL-substring repair guidance is semantically inconsistent.","scores":{"semantic_correctness":83,"false_positive_control":78,"security_trust_boundaries":76,"verification_design":87,"scope_contract_consistency":94,"docs_usability":84},"findings":[{"severity":"H","category":"false_positive_control","file":"skills/e2e-reviewer/scripts/scan.sh","line":1839,"title":"Regular-file rewrites evade scanner identity checks","evidence":"Candidate validation checks only that each path is still a regular file. A same-path regular file can be rewritten or replaced between discovery, tiers, and final validation without detection, allowing a scan to combine different source states or return clean after inspected content changes.","recommendation":"Record and compare no-follow descriptor fingerprints for every candidate before and after all tiers, and scan immutable descriptor-backed snapshots where possible."},{"severity":"H","category":"security_trust_boundaries","file":"skills/playwright-debugger/SKILL.md","line":115,"title":"Artifact download recipe follows pre-existing symlinks","evidence":"The documented recipe runs `mkdir -p playwright-report` directly before downloading. If that path already names a symlink to a directory, `mkdir -p` succeeds and the subsequent download can write outside the repository, contradicting this file's mandatory pre-write validation at lines 36-44.","recommendation":"Provide a bundled descriptor-relative downloader or staging helper that rejects every symlinked component, creates a unique directory safely, and revalidates it immediately before invoking `gh`."},{"severity":"M","category":"semantic_correctness","file":"skills/playwright-test-generator/code-rules.md","line":157,"title":"Substring URL assertion is replaced with exact URL semantics","evidence":"This rule maps `expect(page.url()).toContain(x)` to `await expect(page).toHaveURL(x)`, while the included canonical replacement contract at `skills/e2e-reviewer/references/applying-fixes.md:27` requires `expect.poll(...).toContain(x)` because the original contract is substring matching.","recommendation":"Replace this row with the canonical `await expect.poll(() => page.url()).toContain(x)` form and enforce parity between generation and reviewer replacement tables."}],"limitations":["No commands, tests, scanners, benchmarks, holdouts, or runtime probes were executed.","The packet omits many scripts invoked by `scripts/ci/ci-local.sh`, so their implementation and fail-closed behavior could not be assessed.","Scores evaluate only included contracts and code; they do not establish observed runtime success, benchmark accuracy, release evidence integrity, or behavior of external hosts and tools."],"verdict":"FAIL"}