The SkillForge validator checks how closely a skill package conforms to Anthropic's published skill format specification and produces a score from 0 to 100.
Important disclaimer: The SkillForge quality score reflects conformance to Anthropic's skill format specification. It does not reflect the skill's functionality, usefulness, or whether it will trigger correctly in practice.
# Validate and print the report
skillforge validate ./my-skill
# JSON output (useful for scripts and CI)
skillforge validate ./my-skill --json
# Validation runs automatically on every install
skillforge install ./my-skillValidation also runs automatically when installing a skill. The result is shown if there are any warnings or errors; a clean install shows only the final score.
Feedback is compiler-style: one line per check, with severity label, line number where applicable, and a plain-English message.
pass YAML frontmatter valid
pass name field present
pass description meets length requirement (34 words)
pass SKILL.md line count OK (42 lines)
error line 12: references assets/template.docx but assets/template.docx not found
warn Unknown subdirectory "bin": only scripts/, references/, assets/ are allowed
pass Bundled resources in correct subdirectories
Format conformance score: 70/100
Validated against: skill-format v1.0
Severity labels:
| Label | Meaning |
|---|---|
pass |
Check passed; points awarded |
warn |
Potential issue; points may be deducted, but installation is never blocked |
error |
Check failed; points deducted |
The score is the sum of points awarded by each check. Maximum: 100 points. The weights are derived from the spec, not hand-tuned; see How the weights were derived.
| Check | Max Points | Check ID |
|---|---|---|
| YAML frontmatter present and parseable | 16 | yaml-frontmatter |
name field present and non-empty |
16 | name-present |
name is kebab-case and uses no reserved words |
11 | name-format |
description field present |
16 | description-present |
description meets 30-word minimum |
6 | description-length |
description ≤ 1024 chars and free of XML tags |
11 | description-format |
SKILL.md under 500 lines |
7 | line-count |
| Only allowed subdirectories present | 4 | allowed-subdirs |
No README.md inside the skill folder |
4 | no-readme |
| All referenced resources exist | 7 | referenced-resources |
| Bundled resources in correct subdirectories | 2 | bundled-resources |
| Total | 100 |
The point values are derived from the specification, not hand-picked. Each check is scored on two axes read from Anthropic's skill guide, and the product is normalized to 100:
- Mandate: how strongly the spec requires it: MUST / "required" / "most important" / security rule = 3; SHOULD / recommended = 2; MAY / optional / SkillForge-only heuristic = 1.
- Impact: what breaks if it's violated: breaks loading or triggering = 3; breaks runtime = 2; cosmetic / hygiene = 1.
weight = round( (mandate × impact) / Σ(mandate × impact) × 100 )
| Check | Mandate | Impact | Raw | Weight |
|---|---|---|---|---|
yaml-frontmatter (also the fatal gate) |
3 | 3 | 9 | 16 |
name-present |
3 | 3 | 9 | 16 |
name-format |
3 | 2 | 6 | 11 |
description-present |
3 | 3 | 9 | 16 |
description-length (30-word heuristic) |
1 | 3 | 3 | 6 |
description-format (≤1024, no <>) |
3 | 2 | 6 | 11 |
line-count |
2 | 2 | 4 | 7 |
allowed-subdirs |
2 | 1 | 2 | 4 |
no-readme |
2 | 1 | 2 | 4 |
referenced-resources |
2 | 2 | 4 | 7 |
bundled-resources |
1 | 1 | 1 | 2 |
Why not calibrate against real skills? Public skills are overwhelmingly generated by the skill-creator, so a corpus is sampled on the very property this score measures: conditioning on the outcome. In practice conformance is near-ceiling (observed median ~90, ~0.1% below 50 across ~18k skills), so almost every check passes almost always. Weighting by observed "discrimination" would therefore penalize the most fundamental checks precisely because everyone satisfies them, and a sensitivity sweep over such a compressed distribution gives a false "weights don't matter" signal. A conformance score's ground truth is the spec itself, so the weights come from the spec's own priority language rather than from outcome data.
What it checks: Whether SKILL.md starts with a valid YAML frontmatter block delimited by ---.
Failure is fatal. If frontmatter is missing or unparseable, the skill scores 0 immediately and no other checks run.
Failure conditions:
- File does not start with
--- - Frontmatter has no closing
--- - YAML between the delimiters is syntactically invalid
Example failure output:
error line 1: Missing YAML frontmatter: file must start with ---
Format conformance score: 0/100
What it checks: Whether the name key is present in the frontmatter and has a non-empty value.
Failure condition: name is absent, null, or whitespace-only.
Example failure output:
error line 2: Required field "name" is missing or empty
What it checks: Whether a present name is kebab-case (^[a-z0-9]+(-[a-z0-9]+)*$: lowercase letters, digits, single hyphens) and does not contain a reserved word. claude and anthropic are reserved by Anthropic.
Only runs when name is present. All-or-nothing: both conditions must hold to earn the points.
Example failure output:
error line 2: name "Bad Name" is not kebab-case: use lowercase letters, digits, and hyphens only
error line 2: name contains reserved word "claude": "claude" and "anthropic" are reserved
What it checks: Whether description is present (16 pts) and meets the 30-word minimum (6 pts).
These are two separate point awards:
- 16 pts for being present and non-empty
- 6 additional pts for having 30 or more words
Why 30 words? Undertriggering is a known problem with Claude skills. A short description like "Helps with debugging." doesn't give Claude enough context to know when to invoke the skill. 30 words is a proxy for enough specificity. (This 30-word rule is a SkillForge heuristic, not a spec requirement, which is why it carries the lowest weight of the description checks.)
Word counting: Splits on whitespace (\s+); punctuation counts as part of its word.
Example failure output:
error line 3: description too short (current: 4 words, recommended: 30+)
What it checks: Whether a present description is ≤ 1024 characters and contains no XML angle brackets (< or >). The spec forbids XML tags because the frontmatter is injected into Claude's system prompt, where stray markup could inject instructions.
Only runs when description is present. All-or-nothing.
Example failure output:
error line 3: description exceeds 1024 characters (current: 1180)
error line 3: description contains XML angle brackets (< >): not allowed in frontmatter
What it checks: Whether SKILL.md is under the 500-line limit per Anthropic's spec.
| Lines | Result |
|---|---|
| ≤ 400 | Pass (15 pts) |
| 401-499 | Warning (15 pts, but warns you're close) |
| 500+ | Error (0 pts) |
Example warning output:
warn SKILL.md is 487 lines: approaching 500 line limit
Example error output:
error SKILL.md is 523 lines: exceeds 500 line limit
What it checks: Whether the skill folder contains only scripts/, references/, and assets/ as subdirectories.
Any other subdirectory (e.g., bin/, src/, test/) produces a warning.
Scoring: Partial credit: 2 points deducted per unknown directory, minimum 0. One unknown directory = 2/4 pts.
Example warning output:
warn Unknown subdirectory "bin": only scripts/, references/, assets/ are allowed
warn Unknown subdirectory "src": only scripts/, references/, assets/ are allowed
What it checks: Whether a file named README.md (any case) sits directly inside the skill folder. The spec says all documentation belongs in SKILL.md or references/; a README.md inside the skill folder is disallowed. (A repo-level README alongside the skill folder is fine and not checked.)
Scoring: Warning only: never blocks install, but the 4 points are withheld when a README is present.
Example warning output:
warn README.md should not be inside the skill folder: put docs in SKILL.md or references/
What it checks: Whether files referenced in SKILL.md via markdown links or relative paths actually exist in the skill folder.
What is detected as a reference:
- Markdown links:
[text](path/to/file), including link titles ([text](path "title")) and#anchors(the fragment is stripped before checking) - Markdown images:
 - Inline-code paths to a known dir:
`references/api-patterns.md`(the spec's recommended way to reference bundled resources) - Bare relative paths starting with
scripts/,references/, orassets/:scripts/setup.sh
External URLs are ignored: https:// and http:// links are not checked.
Not detected: dot-relative paths (./scripts/foo.sh) and reference-style link definitions ([ref]: path).
If all references resolve, full 7 pts are awarded. Any broken reference deducts the full 7 pts (the check fails as a whole).
Example error output:
error line 12: references assets/template.docx but assets/template.docx not found
error line 18: references scripts/setup.sh but scripts/setup.sh not found
What it checks: Whether files in scripts/ and references/ are in the right directories (by file extension).
| Rule | Bad example |
|---|---|
No .md, .txt, .pdf in scripts/ |
scripts/README.md (should be in references/) |
No .sh, .py, .js, .ts, .rb in references/ |
references/setup.sh (should be in scripts/) |
Example warning output:
warn File "scripts/README.md (docs should go in references/)" appears to be misplaced
Warnings are never blockers. Installation always proceeds if the user chooses to continue, regardless of score or warnings.
The score is informational. It tells you the quality of the skill's format conformance so you can make an informed decision before installing. A score of 40 is valid; it only means the skill doesn't follow the format closely.
This matches how package managers work: npm install doesn't prevent you from installing a package with deprecation warnings.
| Score | Meaning |
|---|---|
| 90-100 | Excellent; follows the format closely |
| 70-89 | Good; minor issues |
| 50-69 | Fair; some checks failed, review before installing |
| < 50 | Poor; significant format problems |
| 0 | Fatal; missing or invalid frontmatter |
skillforge validate ./my-skill --json{
"skill": "forensics-agent",
"score": 70,
"diagnostics": [
{
"severity": "pass",
"message": "YAML frontmatter valid",
"check": "yaml-frontmatter"
},
{
"severity": "pass",
"message": "name field present",
"check": "name-present"
},
{
"severity": "error",
"line": 12,
"message": "references assets/template.docx but assets/template.docx not found",
"check": "referenced-resources"
}
],
"specVersion": "1.0",
"passCount": 5,
"warnCount": 1,
"errorCount": 1
}ValidationDiagnostic fields:
| Field | Type | Description |
|---|---|---|
severity |
"pass" | "warning" | "error" |
Check result |
line |
number | undefined |
Line number in SKILL.md (1-indexed), if applicable |
message |
string |
Human-readable description |
check |
string |
Machine-readable check ID (see table above) |
The current spec version is 1.0. This is printed at the bottom of every validation report and included in the JSON output as specVersion.
When Anthropic updates the skill format spec, SkillForge will cut a new scorer version. Installed skills track which spec version they were validated against (specVersion in skillforge.json). SkillForge will warn when a skill was validated against an older spec version than the current one.
The validator can be used directly from TypeScript:
import { validateSkill } from 'skillforge/core/validator'
const result = await validateSkill('./forensics-agent')
console.log(result.score) // 91
console.log(result.errorCount) // 0
console.log(result.diagnostics) // ValidationDiagnostic[]The renderValidationReport() function from src/cli/ui/output.ts produces the human-readable report string from a ValidationResult.