Skip to content

[CRAFTING] chore(skills): add audit-widget skill - #2461

Merged
samuelreichert merged 3 commits into
mainfrom
chore/audit-widget-skill
Oct 29, 2026
Merged

samuelreichert merged 3 commits into
mainfrom
chore/audit-widget-skill

Conversation

@samuelreichert

@samuelreichert samuelreichert commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Pull request type

No code changes (changes to documentation, CI, metadata, etc.)


Description

Adds the audit-widget agent skill to .agents/skills/ (also available through the .claude/skills symlink).

What it does: /audit-widget [widget] audits one widget package and writes a point-in-time report to packages/pluggableWidgets/<widget>/AUDIT.md. If no widget is given, it asks which one. The report covers eight categories:

  • unit tests
  • e2e flows
  • performance
  • dependencies: outdated versions and known advisories (pnpm audit, filtered to the widget)
  • security: untrusted data that reaches HTML, URLs, scripts or other dangerous sinks
  • code smells
  • missing behaviours, including accessibility
  • drift between the widget and its mx-docs page

Every finding needs file:line evidence and a concrete failure scenario. The detection heuristics are in checklists.md, and the report format is in report-template.md.

Report only. The skill reads code and runs read-only commands. It does not fix code, install packages, change the lockfile, or commit. Each run is a fresh snapshot that replaces the previous AUDIT.md, so a widget can be audited again every few months or whenever needed.

Lessons from auditing Rich Text 5.1.0 with this skill (#2453, #2454 to #2460), built into the skill:

  • Read the library's code before reporting a framework-behaviour bug. One false positive in that audit came from @tiptap/react callback forwarding.
  • A Critical finding needs proof, such as a failing test or command output. Without it, the finding is reported as High with needs repro.

Configuration: the docs category reads the mx-docs repo from $MX_DOCS_PATH (default ~/Projects/mx-docs). If it is missing, the category is skipped and the report says why. E2E page discovery uses mxcli only when it is already installed.

🤖 Generated with Claude Code

@samuelreichert
samuelreichert requested a review from a team as a code owner October 6, 2026 14:02
@github-actions

This comment has been minimized.

yordan-st
yordan-st previously approved these changes Oct 8, 2026

@yordan-st yordan-st left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@samuelreichert
samuelreichert force-pushed the chore/audit-widget-skill branch from 67a2aea to 209cd1f Compare October 12, 2026 12:55
@github-actions

This comment has been minimized.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@samuelreichert samuelreichert changed the title [CRAFTING] chore(skills): add audit-widget skill with rich text example report [CRAFTING] chore(skills): add audit-widget skill Oct 26, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

AI Code Review

🔶 Changes requested — one or more medium-severity items must be addressed


What was reviewed

File Change
.agents/skills/README.md Added audit-widget to directory listing and usage section
.agents/skills/audit-widget/SKILL.md New skill: invocation, hard rules, 4-step workflow
.agents/skills/audit-widget/checklists.md Detection heuristics for 8 audit categories
.agents/skills/audit-widget/report-template.md AUDIT.md output template with field rules

Skipped (out of scope): dist/, pnpm-lock.yaml

No CHANGELOG entry required — this is a documentation/tooling-only PR with no widget runtime behaviour changes.


Findings

🔶 Medium — process.argv[1] silently suppresses all advisory output

File: .agents/skills/audit-widget/checklists.md line 125
Problem: When running node -e 'script' <widget>, Node.js sets process.argv[1] to "" (the empty string — there is no script file path), not the widget argument. The widget name lands at process.argv[2]. As written, w is always "", so the startsWith(\packages__pluggableWidgets__${w}>`)` filter never matches any advisory path. The command prints nothing for every widget, making the vulnerability filter silently inert.

Fix:

// checklists.md line 125 — change [1] to [2]
const w = process.argv[2], d = JSON.parse(require("fs").readFileSync(0, "utf8"));

🔶 Medium — Snyk security check is failing

File: CI check security/snyk (AppDev AppStore content)
Problem: The Snyk security gate is in a FAILURE state for this PR. While the added files are Markdown, the check should be green (or confirmed as a pre-existing failure on main) before merge. An unexplained red security gate is a merge blocker.
Fix: Investigate the Snyk failure. If it pre-dates this PR and is tracked separately, add a note in the PR description linking the tracking issue.


Positives

  • Hard rules are precise and enforceable — enumerating exactly which pnpm subcommands are forbidden (install, add, up, audit --fix) leaves no ambiguity for an agent; a vague "don't change dependencies" would not.
  • Evidence-first gate in section 3 — requiring a file:line or command output for every finding, and explicitly saying "Drop speculative findings silently," directly encodes the right quality tradeoff for an audit tool.
  • DEP instructions correctly separate commands from search — instructing the agent to run DEP itself (not delegate to a read-only subagent) is the right call given it requires shell commands, and the comment says why.
  • Library false-positive guard — the @tiptap/react example in section 3 is a concrete lesson baked in as a guardrail; this is exactly the kind of non-obvious heuristic that makes skill instructions durable.

@yordan-st yordan-st left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks nice, lets merge and put it to use.

@samuelreichert
samuelreichert merged commit df014e4 into main Oct 29, 2026
20 of 21 checks passed
@samuelreichert
samuelreichert deleted the chore/audit-widget-skill branch October 29, 2026 08:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants