Skip to content

Commit 623388a

Browse files
TMHSDigitalclaude
andcommitted
tests: each block runs as a named section; a throw no longer ends the run (#203)
Both test files gain section(name, fn): the 68 self-contained blocks run through it, a section that throws is reported as a failure and the run continues, and TEST_ONLY=<words> runs only the matching sections. Counts were already real (checks.test counts the checks it sees); lib.test now also reports how many sections ran. Assertion totals are unchanged. Not a migration to node:test: the files share top-level fixtures across 1,800 assertions, and sections give the isolation and selection without rewriting them. Closes #203 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LpkeXQPaUhxjgqdqnk6BrJ
1 parent f904b85 commit 623388a

3 files changed

Lines changed: 173 additions & 139 deletions

File tree

‎CONTRIBUTING.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -73,7 +73,7 @@ To add a starter stack (one pick per need for a kind of game), follow [`stacks/R
7373
Every pull request, every push to `main` (before the site deploys) and every release run one sequence, `npm run check` in [`package.json`](package.json). Pull requests and pushes run it on Node 24, again on Node 22, and on Windows; releases run it on Node 24:
7474

7575
1. `npm run lint` (`site/lint.mjs`): every script parses, and none imports a name it does not use. It is syntax-only, not ESLint: a misspelled variable still passes and fails only when that line runs, so code the tests never reach needs a test or a careful read.
76-
2. `npm test`: the checks against known-bad fixtures (`site/checks.test.mjs`) and the site modules' tests (`site/lib/lib.test.mjs`).
76+
2. `npm test`: the checks against known-bad fixtures (`site/checks.test.mjs`) and the site modules' tests (`site/lib/lib.test.mjs`). Each block runs as a named section: one that throws is reported as a failure and the rest still run, and `TEST_ONLY=<words> npm test` runs only the sections whose name contains those words.
7777
3. `npm run validate` (`site/validate.mjs`): every entry, link, count table and file against the checks.
7878
4. `npm run check-added` (`site/check-added.mjs`): no entry's `added` date is later than the file's first commit, following renames. It needs the full git history; CI fetches it, and a shallow clone of your own will fail this step.
7979
5. `npm run build` (`site/build.mjs`): renders every page and runs the page checks.

‎site/checks.test.mjs‎

Lines changed: 32 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -101,6 +101,23 @@ function rejects(label, errors, needle) {
101101
passed += 1;
102102
}
103103

104+
// Sections (#203): each self-contained block runs as one. A section that
105+
// throws is reported as a failure and the run goes on, so one crash does not
106+
// hide the rest. TEST_ONLY=<words> runs only the sections whose name contains
107+
// them (the few assertions outside any section still run).
108+
const ONLY = String(process.env.TEST_ONLY || "").toLowerCase();
109+
let sectionsRun = 0;
110+
async function section(name, fn) {
111+
if (ONLY && !name.toLowerCase().includes(ONLY)) return;
112+
sectionsRun += 1;
113+
try {
114+
await fn();
115+
} catch (err) {
116+
const where = err && err.stack ? err.stack.split("\n").slice(0, 3).join(" | ") : String(err);
117+
failures.push(`section "${name}" threw: ${where}`);
118+
}
119+
}
120+
104121
function eq(label, actual, expected) {
105122
note(label);
106123
if (actual === expected) passed += 1;
@@ -741,25 +758,25 @@ rejects("V19 rejects a tag that restates the licence", checkValueAliases("bad.md
741758
accepts("V19 keeps a tag that only mentions a licence word", checkValueAliases("ok.md", { tags: ["odbl-adjacent", "free-tier"] }, aliases));
742759

743760
/* Licence families: every licence in exactly one Licence-filter group ---- */
744-
{
761+
await section("Licence families: every licence in exactly one Licence-filter group", async () => {
745762
const fams = Object.entries(vocab.families).filter(([k]) => !k.startsWith("_"));
746763
const homeless = Object.keys(vocab.licenses).filter((l) => fams.filter(([, f]) => f.licenses.includes(l)).length !== 1);
747764
accepts("every licence value sits in exactly one family", homeless);
748765
const unknown = fams.flatMap(([, f]) => f.licenses).filter((l) => !(l in vocab.licenses));
749766
accepts("families name only licence values the vocabulary has", unknown);
750-
}
767+
});
751768

752769
/* V22: maintenance ------------------------------------------------------ */
753770
rejects("V22 rejects an unknown maintenance value", checkMaintenance("bad.md", { maintenance: "dead" }), 'maintenance "dead" is not one of archived | inactive');
754771
accepts("V22 accepts archived", checkMaintenance("ok.md", { maintenance: "archived" }));
755772
accepts("V22 leaves the field optional", checkMaintenance("ok.md", {}));
756-
{
773+
await section("block at line 773", async () => {
757774
const now = Date.parse("2026-09-26T00:00:00Z");
758775
const same = (label, actual, expected) => (actual === expected ? accepts(label, []) : accepts(label, [`got ${actual}, want ${expected}`]));
759776
same("an archived repo is archived", maintenanceFromRepo({ archived: true, pushed_at: "2026-09-01T00:00:00Z" }, now), "archived");
760777
same("no push in over three years is inactive", maintenanceFromRepo({ archived: false, pushed_at: "2023-01-01T00:00:00Z" }, now), "inactive");
761778
same("a recent push is neither", maintenanceFromRepo({ archived: false, pushed_at: "2025-01-01T00:00:00Z" }, now), null);
762-
}
779+
});
763780

764781
/* V23: engine fields ---------------------------------------------------- */
765782
rejects("V23 rejects an unknown engine_lock", checkEngineFields("bad.md", { engine_lock: "unity" }), 'engine_lock "unity" is not one of unreal');
@@ -816,7 +833,7 @@ accepts("V25 ignores a missing name", checkTitleMatchesName("ok.md", {}, "# Anyt
816833

817834
/* Thumbnails: the manifest, the files and the gate must agree ------------- */
818835
import { checkPreviewField, checkThumbManifest } from "./checks.mjs";
819-
{
836+
await section("block at line 836", async () => {
820837
const okEntry = { id: "a", license: "CC0", status: "active", url: "https://a.itch.io/x" };
821838
const manifest = { thumbs: { a: { fetched: "2026-10-01", license: "CC0", hash: "h", width: 640, height: 400, source: "https://a.itch.io/x", image: "https://i/a.png" } } };
822839
accepts("thumbs manifest: file and record agree", checkThumbManifest(manifest, new Map([["a", okEntry]]), ["a.webp"]));
@@ -827,7 +844,7 @@ import { checkPreviewField, checkThumbManifest } from "./checks.mjs";
827844
rejects("thumbs manifest: rejects a thumbnail whose entry is no longer eligible", checkThumbManifest(manifest, new Map([["a", { ...okEntry, license: "custom" }]]), ["a.webp"]), 'thumbnail for "a" must be removed: licence custom is not in the preview allowlist (run node site/thumbs.mjs)');
828845
accepts("thumbs manifest: licence drift between two allowed licences is not an error", checkThumbManifest(manifest, new Map([["a", { ...okEntry, license: "MIT" }]]), ["a.webp"]));
829846
rejects("thumbs manifest: stray file", checkThumbManifest(manifest, new Map([["a", okEntry]]), ["a.webp", "notes.txt"]), "site/public/thumbs/notes.txt is not a <id>.webp thumbnail");
830-
}
847+
});
831848
accepts("preview field: absent", checkPreviewField("ok.md", {}));
832849
accepts("preview field: none", checkPreviewField("ok.md", { preview: "none" }));
833850
accepts("preview field: https url", checkPreviewField("ok.md", { preview: "https://a/b.png" }));
@@ -858,7 +875,7 @@ accepts("V31 accepts a quoted restriction", checkNonCommercialEvidence("ok.md",
858875
rejects("V31 rejects an unquoted restriction", checkNonCommercialEvidence("bad.md", { status: "active", commercial: false }, "## Evidence\n\n- Live page (2026-01-01): \"free download\"\n"), "bad.md is active with commercial: false but no Evidence line mentions the commercial restriction");
859876

860877
/* V1: per-entry basics (validate.mjs) --------------------------------------- */
861-
{
878+
await section("V1: per-entry basics (validate.mjs)", async () => {
862879
const good = {
863880
id: "foo", name: "Foo", url: "https://a.example/", category: "2d", subcategories: ["sprites"], license: "CC0",
864881
commercial: true, attribution_required: false, formats: ["PNG"], tags: ["pixel"], added: "2026-01-01",
@@ -890,10 +907,10 @@ rejects("V31 rejects an unquoted restriction", checkNonCommercialEvidence("bad.m
890907
accepts("V1 featured ids that resolve", checkFeatured(["foo", { id: "bar", need: "x" }], new Set(["foo", "bar"])));
891908
rejects("V1 featured id that does not resolve", checkFeatured([{ id: "nope" }], new Set(["foo"])), 'featured id "nope" does not resolve');
892909
rejects("V1 featured item without an id", checkFeatured([{ need: "x" }], new Set(["foo"])), 'featured id "undefined" does not resolve');
893-
}
910+
});
894911

895912
/* V11: comparison tables with a Licence column ------------------------------ */
896-
{
913+
await section("V11: comparison tables with a Licence column", async () => {
897914
const metas = new Map([
898915
["catalog/3d/ambientcg.md", { license: "CC0" }],
899916
["catalog/environment/poly-haven.md", { license: "CC0" }],
@@ -913,7 +930,7 @@ rejects("V31 rejects an unquoted restriction", checkNonCommercialEvidence("bad.m
913930
accepts("V11 ignores links right of the Licence column", checkComparisonLicences("3d", ["| Take | Licence | Notes |", "| --- | --- | --- |", "| [a](ambientcg.md) | CC0 | see [st](sharetextures.md) |"].join("\n"), metas));
914931
accepts("V11 ignores tables without a Licence column", checkComparisonLicences("3d", ["| Take | Notes |", "| --- | --- |", "| [st](sharetextures.md) | CC0 |"].join("\n"), metas));
915932
accepts("V11 leaves the ID listing to checkCategoryReadmeRows", checkComparisonLicences("3d", ["| ID | Licence |", "| --- | --- |", "| [st](sharetextures.md) | CC0 |"].join("\n"), metas));
916-
}
933+
});
917934

918935
/* V34: preview allowlist agrees with the vocabulary ------------------------ */
919936
accepts("V34 the real allowlist matches the real vocabulary", checkPreviewVocabulary(vocab));
@@ -922,15 +939,15 @@ rejects("V34 rejects a required licence the caption would not credit", checkPrev
922939
rejects("V34 rejects a credit on a licence that does not ask for one", checkPreviewVocabulary({ licenses: { MIT: { attribution: "notice" } } }, ["MIT"], () => true), "carry a credit the licence does not ask for");
923940

924941
/* V38: a waived credit on a BY licence is quoted --------------------------- */
925-
{
942+
await section("V38: a waived credit on a BY licence is quoted", async () => {
926943
const v = { licenses: { "CC-BY-4.0": { attribution: "required" }, MIT: { attribution: "notice" } } };
927944
const meta = { license: "CC-BY-4.0", attribution_required: false };
928945
accepts("V38 accepts a dated waiver quote", checkAttributionWaiver("ok.md", meta, "## Evidence\n\n- Page (2026-11-03): \"You do NOT have to credit me.\"\n", v));
929946
accepts("V38 ignores required credit", checkAttributionWaiver("ok.md", { ...meta, attribution_required: true }, "", v));
930947
accepts("V38 ignores a licence that does not require credit", checkAttributionWaiver("ok.md", { license: "MIT", attribution_required: false }, "", v));
931948
rejects("V38 rejects a waiver stated only in Notes", checkAttributionWaiver("bad.md", meta, "## Notes\n\n- Attribution waived\n\n## Evidence\n\n- Page (2026-08-24): \"CC BY 4.0\"\n", v), "quote the rights holder's waiver");
932949
rejects("V38 rejects an undated waiver line", checkAttributionWaiver("bad.md", meta, "## Evidence\n\n- Page: \"credit is optional\"\n", v), "quote the rights holder's waiver");
933-
}
950+
});
934951

935952
/* V37: AI-generated content stays tagged and listed ------------------------ */
936953
accepts("V37 accepts a tagged entry", checkAiTag("ok.md", { tags: ["ai-generated"] }, "Some sounds are AI-generated."));
@@ -974,7 +991,7 @@ rejects("V32 rejects a missing value", checkLicenceGuide({ licenses: { MIT: {},
974991
rejects("V32 a longer value does not count for its prefix", checkLicenceGuide({ licenses: { "CC-BY": {} } }, "| **CC-BY-4.0** | yes |"), 'does not mention the licence value "CC-BY"');
975992

976993
/* validate's file-level checks, moved here to be tested (#224) ----------- */
977-
{
994+
await section("validate's file-level checks, moved here to be tested (#224)", async () => {
978995
const { parseFrontmatter } = await import("./lib/frontmatter.mjs");
979996
const { EXTERNAL_SCHEME_RE, ANY_SCHEME_RE } = await import("./lib/shared.mjs");
980997
const { markdownAnchors: anchors } = await import("./checks.mjs");
@@ -1020,12 +1037,12 @@ rejects("V32 a longer value does not count for its prefix", checkLicenceGuide({
10201037
eq("form options: no category dropdown", categoryFormOptions("body:\n - type: input\n id: name\n").join(), "");
10211038
eq("form options: quoted options", categoryFormOptions(' - type: dropdown\n id: category\n attributes:\n options:\n - "2d"\n - \'3d\'\n').join(), "2d,3d");
10221039
eq("form options: stop at the next key", categoryFormOptions(" id: category\n attributes:\n options:\n - 2d\n - audio\n validations:\n required: true\n").join(), "2d,audio");
1023-
}
1040+
});
10241041

10251042
/* ----------------------------------------------------------------------- */
10261043
if (failures.length) {
10271044
console.error(`checks.test failed (${failures.length}):`);
10281045
for (const f of failures) console.error(` - ${f}`);
10291046
process.exit(1);
10301047
}
1031-
console.log(`checks.test ok: ${passed} assertions across ${checksSeen.size} checks`);
1048+
console.log(`checks.test ok: ${passed} assertions across ${checksSeen.size} checks${ONLY ? ` (sections matching "${ONLY}")` : ""}`);

0 commit comments

Comments
 (0)