fix(examples): give gn-bundle-roundtrip a falsifier that can fail (#408)

The catalog falsifier expected exit 2 for --legacy-rna, but both outcomes
of that branch returned 2 (5.x rejects the legacy node, and the legacy node
unexpectedly exists), 2 also meant "carrier mesh rewritten", and argparse
exits 2 for an unknown flag. The row passed whether or not the contract
held, and would have passed with the flag deleted.

Each outcome now has its own code: 6 carrier rewritten, 7 legacy node
rejected (the catalog expectation), 8 legacy node accepted. The catalog
lint now rejects expect_exit 2 for every row, since a usage error can
always satisfy it.

Closes #349

Signed-off-by: TMHSDigital <154358121+TMHSDigital@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

Author: TMHSDigital

CLAUDE.md

@@ -177,7 +177,7 @@ The host runner is `tests/smoke/run_example.py`. Shipped examples are listed in
   `$BDT_SMOKE_SIDECAR` (set by the runner). The harness asserts after Blender
   exits. Not a gallery still.
 - **Falsifiers in CI:** every catalog row carries `falsifiers`, at least one
-  `{"args": [...], "expect_exit": N}` (N not 0, 1 or 77) or, for a contract
+  `{"args": [...], "expect_exit": N}` (N not 0, 1, 2 or 77; 2 is argparse usage) or, for a contract
   the sidecar check owns, `{"args": [...], "expect_sidecar_fail": true}`.
   A falsifier that targets a version-specific API change takes
   `"min_version": "5.2"` and is recorded as SKIP below it (its README must

examples/gn-bundle-roundtrip/README.md

@@ -53,7 +53,8 @@ not read at thumbnail.
 **What failure each check would catch:**
 
 - exit 77 — Blender < 5.0 and not `--force-run`
-- exit 2 — `GeometryNodeCombineBundle` on 5.x (`--legacy-rna`)
+- exit 7 — `GeometryNodeCombineBundle` on 5.x (`--legacy-rna`); exit 8 if
+  that legacy identifier is ever accepted again
 - exit 3 — item names do not pair, so Separate yields no geometry
   (`--mismatch` renames the geometry item → 0/0)
 - exit 4 — count matches but the unpacked Scale/Offset did not arrive
@@ -78,10 +79,13 @@ against it. `77` is the smoke skip protocol, not a product check.
 | --- | --- |
 | 0 | Success |
 | 1 | Uncaught exception (FATAL wrapper); `--force-run` on 4.5 lands here |
-| 2 | argparse / usage; carrier mesh rewritten; `--legacy-rna` on 5.x |
+| 2 | argparse / usage |
 | 3 | Evaluated vert/face count off closed form (`--mismatch`) |
 | 4 | Unpacked Scale/Offset x-extent off (`--bypass`, `--pack-scale 1`) |
 | 5 | `bundle_mark` missing or off closed form (`--mismatch-mark`) |
+| 6 | Carrier mesh rewritten |
+| 7 | `--legacy-rna`: 5.x RNA rejects `GeometryNodeCombineBundle` (the catalog falsifier) |
+| 8 | `--legacy-rna`: `GeometryNodeCombineBundle` was accepted (the rejection contract broke) |
 | 77 | `SMOKE_SKIP:` Bundles require Blender 5.0+ |
 
 Measured on 4.5.11 / 5.0.1 / 5.1.2 / 5.2.1:
@@ -91,12 +95,12 @@ Measured on 4.5.11 / 5.0.1 / 5.1.2 / 5.2.1:
 | (none) | 77 | 0 | 0 | 0 |
 | `--force-run` | 1 | 0 | 0 | 0 |
 | `--bypass` | 77 | 4 | 4 | 4 |
-| `--legacy-rna` | 77 | 2 | 2 | 2 |
+| `--legacy-rna` | 77 | 7 | 7 | 7 |
 | `--mismatch` | 77 | 3 | 3 | 3 |
 | `--mismatch-mark` | 77 | 5 | 5 | 5 |
 | `--pack-scale 1` | 77 | 4 | 4 | 4 |
 
 The `blender-smoke` workflow runs the check on Blender 5.2 LTS (5.1 on the
 weekly cron, the `needs-5.1` PR label, or manual dispatch) and skips on 4.5
-LTS. Smoke does not pass `--bypass`, `--legacy-rna`, `--mismatch`,
-`--mismatch-mark`, or `--force-run`.
+LTS. Its catalog falsifier is `--legacy-rna` (expects exit 7). Smoke does
+not pass `--bypass`, `--mismatch`, `--mismatch-mark`, or `--force-run`.

examples/gn-bundle-roundtrip/gn_bundle_roundtrip.py

@@ -133,7 +133,7 @@ def build(pair, match_names, pack_scale, match_mark=True):
 def check(ob):
     if len(ob.data.vertices) != 1:
         print("ERROR: carrier mesh was rewritten", file=sys.stderr)
-        return 2
+        return 6
 
     coords, faces, marks = eval_mesh(ob)
     xs = [c[0] for c in coords]
@@ -225,12 +225,14 @@ def main():
             tree.nodes.new("GeometryNodeCombineBundle")
         except RuntimeError as exc:
             print(f"ERROR: {type(exc).__name__}: {exc}", file=sys.stderr)
-            return 2
+            return 7
+        # A distinct code so the catalog falsifier (expect 7) fails if the
+        # legacy identifier ever comes back, instead of passing on any 2.
         print(
             "ERROR: GeometryNodeCombineBundle existed; 5.x RNA should reject it",
             file=sys.stderr,
         )
-        return 2
+        return 8
 
     ob = build(
         pair=not args.bypass,

tests/check_smoke_catalog.py

@@ -9,7 +9,7 @@
 Schema: only the known row keys are accepted (a typo such as "falsifer" would
 otherwise be ignored and the falsifier silently never run), and every row
 carries at least one falsifier: {"args": [...], "expect_exit": N} with N not
-0, 1 or 77, or {"args": [...], "expect_sidecar_fail": true} on a row that
+0, 1, 2 (argparse usage) or 77, or {"args": [...], "expect_sidecar_fail": true} on a row that
 declares expect_sidecar.
 
     python tests/check_smoke_catalog.py [--root PATH]
@@ -31,7 +31,9 @@
 ROW_KEYS = {"name", "script", "args", "min_version", "expect_file",
             "expect_sidecar", "sidecar_contains", "falsifiers"}
 FALSIFIER_KEYS = {"args", "expect_exit", "expect_sidecar_fail", "min_version"}
-FORBIDDEN_EXITS = {0, 1, 77}
+# 2 is argparse's usage-error exit: an unknown or deleted flag also exits 2,
+# so a falsifier expecting 2 passes even when its flag no longer exists.
+FORBIDDEN_EXITS = {0, 1, 2, 77}
 
 
 def check_schema(row: dict) -> list[str]:
@@ -60,7 +62,7 @@ def check_schema(row: dict) -> list[str]:
         elif has_exit:
             code = f["expect_exit"]
             if not isinstance(code, int) or isinstance(code, bool) or code in FORBIDDEN_EXITS:
-                errors.append(f"{where}: expect_exit {code!r} must be an int other than 0, 1, 77")
+                errors.append(f"{where}: expect_exit {code!r} must be an int other than 0, 1, 2, 77")
         elif not row.get("expect_sidecar"):
             errors.append(f"{where}: expect_sidecar_fail on a row with no expect_sidecar")
         mv = f.get("min_version")

tests/smoke/catalog.json

@@ -65,7 +65,7 @@
     "name": "gn-bundle-roundtrip",
     "script": "examples/gn-bundle-roundtrip/gn_bundle_roundtrip.py",
     "min_version": "5.0",
-    "falsifiers": [{"args": ["--legacy-rna"], "expect_exit": 2}]
+    "falsifiers": [{"args": ["--legacy-rna"], "expect_exit": 7}]
   },
   {
     "name": "exit-pre-sidecar",