Skip to content

Commit 567e316

Browse files
wormeymanclaude
andauthored
test(cliffs): the ore lever OUT OF SAMPLE - precision holds, unknown count is 14 (#84) (#126)
Everything the last three PRs established about the ore rule was measured on [1500,1500], because that is the only region `oracle-vulcanus-cliff-ore-direction` re-runs with the resources off. A rule characterised on one region and never tested on another is fitted until proven otherwise, so the lever was captured for the two regions the entities fixture covers and it never did. | region | resources | ON | OFF | suppressed | | --- | --- | --- | --- | --- | | [0,0] | 945 tungsten-ore | 283 | 283 | 0 | | [-1200,800] | 1047 coal | 387 | 387 | 0 | - PRECISION 1.000 SURVIVES. Our predicate fires on zero cells in both regions, so 1992 resource entities across two fresh regions produce no false positive. This is the arm that would have caught a rule fitted to [1500,1500]. - THE UNEXPLAINED POPULATION IS 14, NOT 11. #123 could only park `106,26.5`, `90,38.5` and `-1050,1022.5` as undetermined because the lever's region did not cover them. It does now, and the ore suppresses nothing there. - Re-confirms #110's attribution - 27 calcite, 4 geyser, 0 tungsten and coal - at far larger scale than the arm that produced it. The non-vacuity check is in the fixture rather than argued: "0 suppressed" is also what a lever that never reached the generator prints, and the OFF arms read back 0 resources against 945 and 1047. What this does NOT establish: neither region contains calcite or a geyser, the only two controls that suppress anything. So this is a PRECISION test, not a recall test - the rule's positive evidence is still one region, and #125's cascade result is still measured on [1500,1500] alone. Breaking the rule would need a region with calcite in it, and the oracle set has none. New fixture is deliberately separate from `oracle-vulcanus-cliff-ore-direction` so regenerating it cannot rewrite ground truth four merged PRs depend on. PROVENANCE entry added. Claude-Session: https://claude.ai/code/session_015jeznBmWiZywUc7cBw9xFU Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 961a6fb commit 567e316

5 files changed

Lines changed: 19351 additions & 0 deletions

File tree

docs/noise/vulcanus-cliffs-NOTES.md

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3169,3 +3169,61 @@ Nothing shipping. The renderer rejects at the crossing stage and does not run
31693169
`applyCliffConnections` at all - see `## The queue has a CONSUMER` for why that
31703170
is deliberate. This is a statement about the RULE, and about which of its
31713171
remainders now have a mechanism.
3172+
3173+
## The ore lever, OUT OF SAMPLE: precision holds, and the unknown count is 14 (2026-08-03, #84)
3174+
3175+
Everything the last three sections established about the ore rule was measured on
3176+
`[1500,1500]`, because that is the only region `oracle-vulcanus-cliff-ore-direction`
3177+
re-runs with the resources off. **A rule characterised on one region and never
3178+
tested on another is fitted until proven otherwise**, so the lever was captured
3179+
for the two regions the entities fixture covers and it never did.
3180+
3181+
`test/cliffOreLeverOutOfSample.spec.ts`,
3182+
`oracle-vulcanus-cliff-ore-direction-regions.seed123456.json` (a SEPARATE fixture,
3183+
so regenerating it cannot rewrite ground truth four merged PRs depend on).
3184+
3185+
| region | resources present | cliffs ON | cliffs OFF | suppressed |
3186+
| --- | --- | --- | --- | --- |
3187+
| `[0,0]` | 945 tungsten-ore | 283 | 283 | **0** |
3188+
| `[-1200,800]` | 1047 coal | 387 | 387 | **0** |
3189+
3190+
### Precision 1.000 survives
3191+
3192+
Our predicate fires on **zero** cells in both regions. 1992 resource entities
3193+
across two fresh regions produce no false positive, so the rule is not merely
3194+
right on the region it was built from. This is the arm that would have caught a
3195+
rule fitted to `[1500,1500]`, and it did not fire.
3196+
3197+
It also re-confirms #110's per-control attribution - 27 calcite, 4 geyser, **0
3198+
tungsten and coal** - at a far larger scale than the arm that produced it. Both
3199+
of these regions happen to contain only the two resources that suppress nothing.
3200+
3201+
### The unexplained population is 14, not 11
3202+
3203+
#123 could only record that the lever's region did not cover `106,26.5`,
3204+
`90,38.5` and `-1050,1022.5`, and parked them as undetermined. The lever covers
3205+
them now and the ore suppresses nothing there, so they are not ore:
3206+
3207+
| of the 25 missed destructions | count |
3208+
| --- | --- |
3209+
| ORE, by the lever | 11 |
3210+
| **unknown** | **14** |
3211+
3212+
Anywhere those were quoted as "11 unknown + 3 undetermined", the number is 14.
3213+
3214+
### The non-vacuity check is in the fixture, not in an argument
3215+
3216+
"0 suppressed" is exactly what a lever that never reached the generator prints.
3217+
The OFF arms read back **0** resources against 945 and 1047, so the
3218+
`autoplace_controls` override provably applied. That check is the reason the zero
3219+
is worth anything, and it is the standing lesson from #111.
3220+
3221+
### What this does NOT establish
3222+
3223+
**Neither region contains calcite or a geyser**, which are the only two controls
3224+
that suppress anything. So this is a **precision** test, not a recall test: the
3225+
rule's positive evidence is still one region, and the cascade result from the
3226+
section above is still measured on `[1500,1500]` alone. A region with calcite in
3227+
it would be the test that could actually break the rule; there is not one in the
3228+
oracle set, and getting one means choosing a new region rather than re-running an
3229+
existing one.
Lines changed: 197 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,197 @@
1+
import { describe, expect, it } from "vite-plus/test";
2+
3+
import regions from "./fixtures/oracle-vulcanus-cliff-ore-direction-regions.seed123456.json";
4+
import { CLIFF_CODE_TO_ORIENTATION } from "../src/noise/cliffs/cliffCatalog";
5+
import { cliffCodeForOrientation } from "../src/noise/cliffs/cliffConnections";
6+
import { makeCliffPlacementFromFields } from "../src/noise/cliffs/cliffPlacement";
7+
import {
8+
VULCANUS_CLIFF_ELEVATION_0,
9+
VULCANUS_CLIFF_ELEVATION_INTERVAL,
10+
VULCANUS_CLIFF_SMOOTHING,
11+
makeVulcanusCliffFields,
12+
} from "../src/noise/cliffs/vulcanusCliffFields";
13+
import { makeVulcanusOreRejection } from "../src/noise/cliffs/vulcanusOreRejection";
14+
import { buildResources } from "../src/noise/preview/renderVulcanusResources";
15+
import { withCtxDefaults } from "../src/noise/eval/ctx";
16+
17+
/**
18+
* **The ore rule, tested OUT OF SAMPLE for the first time - and precision 1.000
19+
* survives** (#84).
20+
*
21+
* Everything known about the ore -> cliff rule was measured on `[1500,1500]`,
22+
* because that is the only region `oracle-vulcanus-cliff-ore-direction` re-runs
23+
* with the resources off. Three merged results rest on that one region: #123's
24+
* split of the 25 missed destructions into 11 ore and 11 unknown, #125's finding
25+
* that the `onDestroy` cascade closes 4 of the 10 remainders, and precision
26+
* 1.000. **A rule characterised on one region and never tested on another is
27+
* fitted until proven otherwise.**
28+
*
29+
* `oracle-vulcanus-cliff-ore-direction-regions` adds the paired ON /
30+
* ALL-resources-OFF arms for the two regions the entities fixture covers and the
31+
* lever never did, at real cliff settings.
32+
*
33+
* | region | resources present | cliffs ON | cliffs OFF | suppressed |
34+
* | --- | --- | --- | --- | --- |
35+
* | `[0,0]` | 945 tungsten-ore | 283 | 283 | **0** |
36+
* | `[-1200,800]` | 1047 coal | 387 | 387 | **0** |
37+
*
38+
* Two things follow, and one of them changes a merged count.
39+
*
40+
* **Precision holds.** Our predicate fires on **zero** cells in both regions, so
41+
* 1992 resource entities across two fresh regions produce no false positive. The
42+
* rule is not merely right on the region it was built from.
43+
*
44+
* **The three "undetermined" missed destructions are NOT ore.** #123 could only
45+
* say that the lever's region did not cover `106,26.5`, `90,38.5` and
46+
* `-1050,1022.5`. It does now, and the ore suppresses nothing there, so those
47+
* three join the unexplained population: **11 ore, 14 unknown**, not 11/11/3.
48+
*
49+
* It also re-confirms #110's per-control attribution - 27 calcite, 4 geyser,
50+
* **0 tungsten and coal** - at a far larger scale than the arm that produced it.
51+
*
52+
* **The non-vacuity check is in the fixture rather than argued.** "0 suppressed"
53+
* is also what a lever that never reached the generator would print. The OFF
54+
* arms read back **0** resources against 945 and 1047, so the override provably
55+
* applied.
56+
*
57+
* What this does NOT establish: that the rule would hold on a region containing
58+
* calcite or geysers other than `[1500,1500]`. Neither of these two has any, so
59+
* the rule's POSITIVE evidence is still one region. This is a precision test,
60+
* not a recall test.
61+
*/
62+
63+
const INPUT = { seed0: 123456, startingPositions: [{ x: 0, y: 0 }] };
64+
const ctx = withCtxDefaults(INPUT);
65+
const fields = makeVulcanusCliffFields(ctx);
66+
const resources = buildResources(ctx);
67+
const oreRejects = makeVulcanusOreRejection(resources, ctx.vulcanusResourceControls);
68+
const K = (x: number, y: number): string => `${String(x)},${String(y)}`;
69+
const BANDS = {
70+
elevation0: VULCANUS_CLIFF_ELEVATION_0,
71+
interval: VULCANUS_CLIFF_ELEVATION_INTERVAL,
72+
smoothing: VULCANUS_CLIFF_SMOOTHING,
73+
};
74+
75+
interface Ent {
76+
x: number;
77+
y: number;
78+
name: string;
79+
orientation?: string;
80+
}
81+
interface Region {
82+
x0: number;
83+
y0: number;
84+
x1: number;
85+
y1: number;
86+
}
87+
const cases = regions.cases as unknown as {
88+
label: string;
89+
region: Region;
90+
cliffs: Ent[];
91+
resources: Ent[];
92+
}[];
93+
94+
interface Pair {
95+
name: string;
96+
region: Region;
97+
on: Set<string>;
98+
off: Set<string>;
99+
resourceCounts: Record<string, number>;
100+
offResourceCount: number;
101+
ourFires: number;
102+
}
103+
104+
const PAIRS: Pair[] = (() => {
105+
const out: Pair[] = [];
106+
for (let i = 0; i < cases.length; i += 2) {
107+
const on = cases[i];
108+
const off = cases[i + 1];
109+
const r = on.region;
110+
const inR = (p: { x: number; y: number }): boolean =>
111+
p.x >= r.x0 && p.x < r.x1 && p.y >= r.y0 && p.y < r.y1;
112+
const set = (c: (typeof cases)[number]): Set<string> =>
113+
new Set(
114+
c.cliffs.filter((e) => e.name === "cliff-vulcanus" && inR(e)).map((e) => K(e.x, e.y)),
115+
);
116+
const counts: Record<string, number> = {};
117+
for (const q of on.resources) counts[q.name] = (counts[q.name] ?? 0) + 1;
118+
119+
let fires = 0;
120+
for (const p of makeCliffPlacementFromFields(fields, BANDS)
121+
.placedCells(r.x0 - 64, r.y0 - 64, r.x1 + 64, r.y1 + 64)
122+
.filter(inR)) {
123+
const o = CLIFF_CODE_TO_ORIENTATION[p.code];
124+
if (o !== undefined && oreRejects(cliffCodeForOrientation(o), p.x, p.y)) fires++;
125+
}
126+
out.push({
127+
name: on.label.replace(", resources ON", ""),
128+
region: r,
129+
on: set(on),
130+
off: set(off),
131+
resourceCounts: counts,
132+
offResourceCount: off.resources.length,
133+
ourFires: fires,
134+
});
135+
}
136+
return out;
137+
})();
138+
139+
describe("the ore lever on the two regions it had never covered", () => {
140+
/**
141+
* **The non-vacuity arm, first, because "0 suppressed" is also what a lever
142+
* that never reached the generator prints.** The OFF arms read back zero
143+
* resources against 945 and 1047 - so the `autoplace_controls` override did
144+
* apply, and the zero below is a measurement.
145+
*/
146+
it("proves the override reached the generator", () => {
147+
expect(PAIRS.map((p) => p.name)).toEqual(["[0,0]", "[-1200,800]"]);
148+
expect(PAIRS.map((p) => p.resourceCounts)).toEqual([{ "tungsten-ore": 945 }, { coal: 1047 }]);
149+
expect(PAIRS.map((p) => p.offResourceCount)).toEqual([0, 0]);
150+
});
151+
152+
/**
153+
* **Zero cliffs move in either region**, in either direction - which also
154+
* re-confirms #99's one-way property and #110's attribution of the 31 to
155+
* calcite and geyser with nothing from tungsten or coal, now against 1992
156+
* resource entities rather than the handful that arm carried.
157+
*/
158+
it("finds the ore suppresses nothing outside [1500,1500]", () => {
159+
for (const p of PAIRS) {
160+
expect([...p.off].filter((k) => !p.on.has(k))).toEqual([]);
161+
expect([...p.on].filter((k) => !p.off.has(k))).toEqual([]);
162+
}
163+
expect(PAIRS.map((p) => p.on.size)).toEqual([283, 387]);
164+
expect(PAIRS.map((p) => p.off.size)).toEqual([283, 387]);
165+
}, 300000);
166+
167+
/**
168+
* **Precision 1.000 survives out of sample.** Our predicate fires on zero
169+
* cells in both regions, so it invents no rejection where the game has none.
170+
* This is the arm that would have caught a rule fitted to `[1500,1500]`.
171+
*/
172+
it("fires on no cell in either region", () => {
173+
expect(PAIRS.map((p) => p.ourFires)).toEqual([0, 0]);
174+
}, 300000);
175+
176+
/**
177+
* **The three missed destructions #123 had to leave undetermined are not
178+
* ore.** They sit in these two regions, and the lever now covers them, so the
179+
* unexplained population is **14**, not 11 with 3 unknown-status.
180+
*/
181+
it("resolves the three cells #123 could not attribute", () => {
182+
const undetermined = ["106,26.5", "90,38.5", "-1050,1022.5"];
183+
for (const k of undetermined) {
184+
const p = PAIRS.find((q) => {
185+
const [xs, ys] = k.split(",");
186+
const x = Number(xs);
187+
const y = Number(ys);
188+
return x >= q.region.x0 && x < q.region.x1 && y >= q.region.y0 && y < q.region.y1;
189+
});
190+
expect(p).toBeDefined();
191+
// Absent with the resources ON and absent with them OFF: the game destroys
192+
// it either way, so no resource is responsible.
193+
expect(p?.on.has(k)).toBe(false);
194+
expect(p?.off.has(k)).toBe(false);
195+
}
196+
}, 300000);
197+
});

test/fixtures/PROVENANCE.json

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -311,6 +311,10 @@
311311
"tree-expressions.2.1.11.json": {
312312
"factorioVersion": "2.1.11",
313313
"evidence": "the version is in the filename, and in the commit that added it"
314+
},
315+
"oracle-vulcanus-cliff-ore-direction-regions.seed123456.json": {
316+
"factorioVersion": "2.1.12",
317+
"evidence": "captured 2026-08-03 by test/oracle/capture.ts vulcanus-cliff-ore-direction-regions against the installed binary, which pnpm refs:sync --check reported in sync at 2.1.12 at capture time. The OUT-OF-SAMPLE arm for the ore -> cliff rule: oracle-vulcanus-cliff-ore-direction only re-runs [1500,1500], so every quantity known about the rule was characterised on one region. Four arms, paired ON / ALL-resources-OFF at REAL cliff settings, over the two regions the entities fixture covers and the lever never did - [0,0] {0,0,256,256} and [-1200,800] {-1200,800,-944,1056}. Each case records the autoplace controls the SURFACE reported back and dumps the cliffs, the resources and the prototype collision geometry from one generated surface, so an override that failed to apply cannot be mistaken for a term that does not matter - and the OFF arms do read back 0 resources against 945 and 1047, which is the non-vacuity check. Deliberately a separate file so regenerating it cannot rewrite ground truth that merged work depends on."
314318
}
315319
}
316320
}

0 commit comments

Comments
 (0)