raw.json

{"summary":"The packet defines extensive framework semantics, fail-closed verification contracts, and trust gates, but three material defects block a strong-quality verdict: a Playwright fix recommends a Jest-DOM matcher, the scanner silently excludes all public directories, and debugger write examples lack an executable path-safety guard.","scores":{"semantic_correctness":78,"false_positive_control":80,"security_trust_boundaries":82,"verification_design":88,"scope_contract_consistency":86,"docs_usability":85},"findings":[{"severity":"H","category":"semantic_correctness","file":"skills/e2e-reviewer/references/applying-fixes.md","line":29,"title":"Playwright fix uses a Jest-DOM matcher","evidence":"The Playwright replacement table maps getByText(...).toBeTruthy() to toBeInTheDocument(), although Playwright getByText returns a Locator and the packet's authoritative #4f contract instead requires an awaited Playwright locator assertion such as toBeVisible(). Applying this row can generate an unsupported matcher or require unrelated Jest-DOM setup.","recommendation":"Move this row exclusively to the RTL/Jest/Vitest section and use await expect(locator).toBeVisible() for proven Playwright Locator subjects."},{"severity":"H","category":"false_positive_control","file":"skills/e2e-reviewer/scripts/scan.sh","line":1438,"title":"Blanket public-directory exclusion can produce false-clean scans","evidence":"The shared exclusion predicate treats every path under public as scanner-excluded. Consequently valid Playwright or Cypress source placed there is removed from tree validation and all detection tiers, despite the public contracts describing generated/vendor/report/eval exclusions rather than an unconditional public source boundary.","recommendation":"Remove the blanket public exclusion, or restrict it to proven generated assets and document the exact exclusion consistently across scanner and public scope contracts."},{"severity":"H","category":"security_trust_boundaries","file":"skills/playwright-debugger/SKILL.md","line":86,"title":"Report merge example writes through unchecked paths","evidence":"The documented merge command creates playwright-report and redirects output to playwright-report/results.json directly. Earlier prose requires rejecting symlinked roots and destinations, but the packet supplies no executable validation step before this write, so following the example can overwrite a symlink target outside the intended report root.","recommendation":"Provide and invoke a bundled descriptor-relative safe-output helper before mkdir and redirection, or have a helper create and atomically publish the validated destination without shell redirection."}],"limitations":["No commands or benchmarks were run; scores assess only the frozen contracts and included implementations.","Holdouts, raw benchmark evidence, repository history, and approximately 41 KB of source were omitted and were not inferred.","Many CI targets referenced by ci-local.sh are absent from the packet, so their implementations and actual outcomes cannot be assessed.","The README intentionally excludes several sections, limiting assessment of those public explanations."],"verdict":"FAIL"}