feat(fabrika): land skill-doctor — byte-exact upstream vendor, fabrika-shaped (#8048) - #8063
Draft
creosB wants to merge 3 commits into
Draft
feat(fabrika): land skill-doctor — byte-exact upstream vendor, fabrika-shaped (#8048)#8063creosB wants to merge 3 commits into
creosB wants to merge 3 commits into
Conversation
…a-shaped (#8048) Vendors Warp Skill Doctor (MIT, warpdotdev/common-skills @ b811c243) byte-exact (blob-SHA verified): scorers, references, collectors, renderer, assets, both unittest suites, LICENSE. Upstream's SKILL.md is deliberately not vendored — SKILL.md is authored fresh as the fabrika routing surface (conventions §1/§2) with contract.md carrying collector flags, the scoring contract, and report artifacts by section (ADR 0296). PROVENANCE.md carries the byte-exact/editable split and the re-copy re-sync rule. A new CI job runs both upstream suites (#8048 §8); .gitignore covers the skill's results dir and Python bytecode. Inert on arrival: opencode collector is #8049, corpus discovery #8051.
Contributor
No preview deploy
|
…nformance (#8048) Proposed test adaptation awaiting maintainer acceptance. Target 1 runs both unchanged upstream suites in an isolated temp tree against upstream's own SKILL.md, downloaded from the pinned commit b811c243 and hash-gated before use — the shipped fabrika SKILL.md never enters that tree. Target 2 runs a new fabrika-authored conformance suite (13 doc-level tests, PROVENANCE-listed) over the shipped SKILL.md/contract.md: session scoping, scoring contract with label-to-score mappings and aggregation weights, failed-conversation threshold and edit gate, section addressability, documented limitations, §4 literal commands. Workflow gains shell: bash + set -euo pipefail; explicit guards for download failure, blob mismatch, and conformance-file isolation. Replacement verified locally; PR CI pending.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Landing child of #8035 (epic). Vendors the Warp Skill Doctor (MIT,
warpdotdev/common-skills@b811c243) byte-exact intoclaude-plugins/fabrika/skills/skill-doctor/, already shaped to skill-conventions. Status: tests/lint passed; leak scan ran and failed on path-policy matches — the scan refused two doc surfaces on~/.claudepath literals:contract.md(sanitized ine715358e) and the byte-exact upstreamreferences/supported-harnesses.md(held as vendored; its refusal stands until upstream changes the text). The test adaptation is a proposal awaiting maintainer acceptance (see "Proposed adaptation" below).Byte-exact vendor — git blob SHAs vs upstream tree @
b811c24365aeassets/pierre-diffs.js1ecc2c99…a25400538a97c2b21a2a172dassets/warp-pixel-icon.svg0ed8d084…7aeb0e46ec53references/skill-improvements.mdbbf1dace…f1661d4freferences/supported-harnesses.mde8f589b7…316e37ec50scorers/code-quality.md59179f8d…535c5afb6c5b14c37ascorers/efficiency.md28648134…019a8284bbe9a04scripts/collect_sessions.pyf24fb541…2aa335835053scripts/render_report.py972da3be…4c4ed5a2ab6fscripts/test_collect_sessions.pyf659ac6e…b4ce4244dscripts/test_render_report.py46658d0f…5153dcb6c3c30204scripts/warp_decoder.pyc54cebdd…269e0aac11c5498b1fLICENSE00bd0da9…4ba8565fc(upstream repo-root MIT)Upstream's own
SKILL.md(9fd7d77d…f598d) is deliberately not vendored — per conventions §1/§2 the landing authors a fabrika routing surface instead. Full SHAs inPROVENANCE.md.Fabrika-authored (editable per
PROVENANCE.md)SKILL.md— routing surface, user-invoked (containment-exempt: maintainer-by-hand), §4 plain-literal commands (the POSIXmktemprecipe does not survive), states the inert-on-arrival contract: zero sessions expected until opencode collector: skill-doctor reads phoenix's real session history #8049 (opencode collector),--skills-dirrequired until skill-doctor discovers fabrika's skills corpus with no flag #8051.contract.md— collector flags, scoring contract, report artifacts; each sectionwire doc-section-addressable (verified).PROVENANCE.md— the byte-exact/editable split and the re-copy re-sync rule.CI — replacement verified locally; PR CI pending
The workflow now runs two separate test targets; the job fails if either fails. No skips, no failure allowlists, no continue-on-error, no output parsing.
SKILL.md, downloaded from the pinned commitb811c24365ae505bfc9646458957b886e29110b5and hash-verified (9fd7d77d…) before use. The shipped fabrikaSKILL.mdnever enters that tree. Verified locally: 14/14 + 11/11 green, including the three SKILL.md-coupled packaging tests, against upstream's own artifact.scripts/test_fabrika_conformance.py(fabrika-authored, PROVENANCE-listed) runs 13 doc-level tests over the shippedSKILL.md/contract.md: session scoping, the scoring contract with label-to-score mappings and aggregation weights, the failed-conversation threshold and edit gate, section addressability, documented collector/discovery limitations, and §4 plain-literal commands. Verified locally: 13/13 green. Doc-level only — it proves the contracts exist where the skill reads them, not runtime behavior.Failure propagation demonstrated locally: a deliberately broken collector fails the step before the renderer runs (renderer verified passing standalone in the same broken tree); wrong pinned SHA, download 404, and a conformance file planted in the baseline tree each fail their guards.
guard skill-lint: verified by CIThe gate fails closed with zero-scope on the filer's Windows machine — the identical red reproduces on untouched
main, so it is a local walker limitation, not this diff. Manual gate-1/3/4 greps over the new directory are clean; frontmatter hand-verified. The authoritative run on ubuntu (this PR's CI, 2026-09-06) passed..gitignoreRows for the skill's
results/directory (transcripts + rendered report embed transcript-derived content and machine-local paths — never committed) and for__pycache__/(Python enters the repo with this child).Proposed adaptation — awaiting maintainer acceptance
The two-target design above is the proposed adaptation, landed in commit
70dad2a8for review: upstream's packaging assertions run against upstream's own artifact (baseline target), while fabrika-authored conformance tests check the replacement surface (target 2). Acceptance is the maintainer's call: if accepted, the baseline target supersedes running the three upstream packaging assertions against the replacedSKILL.mdin place; if rejected, the alternatives (upstream path-parameter patch, or vendoring upstream's startup/branding copy into the fabrika body) come back into scope.