Skip to content

Commit 758c3cb

Browse files
wormeymanclaude
andauthored
test(cliffs): exonerate the tile resolver as issue #18's residual (#74)
The lava collision rejection (#73) costs 10 real cliffs, and the guess recorded alongside it was that the Vulcanus tile resolver - ~98.2% on the 19-way tile name - is worse at a lava boundary. Measured now, on the two committed fixtures, with no Factorio run needed. The guess was wrong in its premise and right in its conclusion. The binary lava/not classification, which is the only thing `tryToAddCliff` reads, is EXACT: 381 oracle positions, 49 lava and 332 not, zero mismatches in either direction. All 7 name errors are non-lava/non-lava confusions inside one biome family, and the 42 positions sitting directly on a lava boundary are 42/42 correct on the full name. It is still off by about a tile, and every real cliff proves it: the game ran this same rejection and kept the cliff, so it saw no lava in that box. 10 of the 1400 real cliffs we place contradict us (0.71%), and in all 10 the offending tile sits at Chebyshev depth 1 inside our own lava. Depth 1 alone does not discriminate, so the control is region [1500,1500]'s 173 correct rejections, spread across the whole range (65 at depth 1, 52 deeper than 6). That rules the resolver out as a cause of the remaining over-placement, which is why it was worth measuring before reading more of generateCliffs. An under-calling resolver would leave false positives next to lava; they are not. Only 4.2% of region [0,0]'s 95 surviving false positives come within 2 tiles of lava against 7.2% of its matched true positives - the wrong way round - and elsewhere the bulk (42/62, 20/29) are more than 8 tiles from any lava. Adds a zero-mismatch pin on the lava classification, with its sensitivity measured rather than assumed: planted scale factors on lava's probability show 1.02 and 1.2 pass while 2, 5 and 20 fail, so it guards against a broken range expression and not against a sub-tile shift. Says so at the assertion. No production code changes. Claude-Session: https://claude.ai/code/session_013eczw9uoWDfN6Wc6kA7UBY Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent ff4787b commit 758c3cb

3 files changed

Lines changed: 131 additions & 12 deletions

File tree

docs/noise/cliffs-NOTES.md

Lines changed: 48 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -164,7 +164,7 @@ extracted from the low word only.
164164
Also note the address above (`0x1016067a0`) is from the build this file was first
165165
written against; under 2.1.12 the symbol is at `0x10160c3ac`. Re-derive from `nm`.
166166

167-
### The collision rejection - NOT ported, and it is issue #18's residual
167+
### The collision rejection - PORTED 2026-07-30 (PR #73), and it is half of issue #18
168168

169169
`EntityMapGenerationTask::tryToAddCliff` (`0x101625038` in 2.1.12) does not just
170170
record the cell. It switches on the `CliffOrientation` (a 20-entry jump table),
@@ -209,13 +209,53 @@ the rule with the real per-orientation boxes and our own tile resolver:
209209
collapsing to 0.65 / 0.70. The real arm rejects almost only false positives.
210210

211211
So it explains region `[1500,1500]`'s over-placement essentially in full and
212-
**does not** explain regions `[0,0]` or `[-1200,800]`, which barely move. It also
213-
costs 6 true positives at `[0,0]`; the tile resolver is ~98% accurate overall and
214-
plausibly worse at a lava boundary, but that is a hypothesis, not a measurement.
215-
216-
The rejection itself is **not implemented** - only the tables it needs are. Wiring
217-
it in gives the cliff overlay a dependency on the Vulcanus tile resolver, which is
218-
a render-cost decision, so it is deliberately a separate change.
212+
**does not** explain regions `[0,0]` or `[-1200,800]`, which barely move.
213+
214+
#### The 10 true positives it costs are a one-tile boundary, and the resolver is not the residual
215+
216+
Wiring the rejection in gives the cliff overlay a dependency on the Vulcanus tile
217+
resolver, so the obvious next suspect was that resolver: it is ~98.2% accurate on
218+
the 19-way tile name, and the natural guess (written into this file and into
219+
`vulcanusCliffEntities.spec.ts` when the rule landed) was that it is worse at a
220+
lava boundary. **Measured 2026-07-30. The guess was wrong in its premise and
221+
right in its conclusion, and neither half needed a Factorio run** - both fixtures
222+
are committed.
223+
224+
- **The binary lava classification is EXACT.** `tryToAddCliff` only ever asks
225+
whether a tile carries `water_tile`, never which tile it is, so the 19-way
226+
argmax is the wrong thing to have been quoting. On all 381 positions of
227+
`oracle-vulcanus-tile-names.seed123456` - 49 lava/lava-hot, 332 not - the
228+
resolver lands on the correct side in **both** directions, zero mismatches.
229+
All 7 name errors are non-lava/non-lava confusions inside one biome family.
230+
Nor is it worse near a boundary: the 42 positions at Chebyshev distance 1 from
231+
a tile of the opposite lava-ness are 42/42 correct on the full name.
232+
`test/vulcanusTiles.spec.ts` now pins the zero.
233+
- **But every real cliff is a negative-space oracle, and 10 of them contradict
234+
us.** The game ran this same rejection and kept the cliff, so the game sees no
235+
lava in that box. Over the 1400 real cliffs we place across the three regions,
236+
10 boxes hold our lava - **0.71%** - and in all 10 the offending tile sits at
237+
Chebyshev depth **1** inside our own lava, on its perimeter. Not one is deep.
238+
Depth 1 is the common case for any lava tile, so that alone does not
239+
discriminate; the control is region `[1500,1500]`'s 173 correct rejections,
240+
which spread across the whole range (65 at depth 1, 52 deeper than 6). The
241+
rule's real work is nowhere near the boundary.
242+
243+
**This rules the resolver out as a cause of the remaining over-placement**, which
244+
is why it was worth measuring before reading any more of `generateCliffs`. A
245+
resolver that UNDER-called lava would leave false positives sitting next to lava.
246+
They do not: of region `[0,0]`'s 95 surviving false positives only 4.2% come
247+
within 2 tiles of any lava, against 7.2% of its 222 matched true positives - the
248+
wrong way round - and in regions `[1500,1500]` and `[-1200,800]` the bulk (42/62
249+
and 20/29) are more than 8 tiles from the nearest lava tile. Whatever remains is
250+
not a lava question, and it is not a tile question.
251+
252+
One cheap fixture improvement fell out of this and has **not** been done: the
253+
cliff dump records only `{x, y, name}`. Adding each entity's `cliff_orientation`
254+
would turn the fixture into a direct oracle for `CLIFF_CODE_TO_ORIENTATION` end
255+
to end - today that table is validated only against the binary's own jump table -
256+
and would give the true collision box for the ~60 real cliffs per region we miss
257+
entirely, which is currently unobtainable because we have no orientation for a
258+
cell we never place.
219259

220260
#### `rotbb` boxes
221261

test/vulcanusCliffEntities.spec.ts

Lines changed: 34 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -122,10 +122,40 @@ describe("Vulcanus cliff placement vs find_entities", () => {
122122
// **Reported rather than smoothed over: recall gets WORSE in two regions.**
123123
// Region 0 loses 6 true positives (0.806 -> 0.784) and region 1 loses 4.
124124
// Those are cells where the game placed a cliff and our tile resolver puts
125-
// lava inside its box. The resolver is ~98.2% accurate overall
126-
// (`vulcanusTiles.spec.ts`) and is plausibly worse at a lava boundary, but
127-
// that is a hypothesis and has not been measured - it is the first thing to
128-
// check before concluding the collision geometry is wrong.
125+
// lava inside its box.
126+
//
127+
// **Those 10 are a one-tile boundary error, and that is measured now
128+
// (2026-07-30), not assumed.** This comment used to carry the guess that
129+
// the resolver "is plausibly worse at a lava boundary"; the guess was
130+
// wrong in its premise and right in its conclusion, so both halves are
131+
// worth stating:
132+
//
133+
// - The resolver is NOT worse at a lava boundary. Its binary lava/not
134+
// classification - the only thing `tryToAddCliff` reads - is EXACT on
135+
// all 381 oracle positions, 49 lava and 332 not, in both directions
136+
// (`vulcanusTiles.spec.ts` now pins this at zero mismatches). The 42
137+
// positions sitting directly on a lava boundary are 42/42 correct even
138+
// on the full 19-way name.
139+
// - It is nonetheless off by about one tile SOMEWHERE, because each real
140+
// cliff the game placed is itself a negative-space oracle: the game ran
141+
// this same rejection and kept the cliff, so the game sees no lava in
142+
// that box. Over the 1400 real cliffs we place across the three regions,
143+
// 10 boxes contradict that - 0.71% - and in **all 10** the offending
144+
// tile sits at Chebyshev depth 1 inside our lava, i.e. on our own
145+
// perimeter. Not one is deep water.
146+
//
147+
// Depth 1 is the common case for any lava tile, so that alone would not
148+
// discriminate; region 1's 173 correct rejections are the control and they
149+
// are spread right across the range (65 at depth 1, 52 deeper than 6). The
150+
// rule's real work is untouched by the boundary.
151+
//
152+
// **This also rules the resolver out as a cause of the residual below**,
153+
// which was the reason to measure it first. An under-calling resolver
154+
// would leave false positives sitting next to lava; they do not. Of region
155+
// 0's 95 surviving false positives only 4.2% come within 2 tiles of any
156+
// lava, against 7.2% of its matched true positives - the wrong way round -
157+
// and in regions 1 and 2 the bulk (42/62 and 20/29) are more than 8 tiles
158+
// from the nearest lava tile. Whatever is left is not a lava question.
129159
//
130160
// A control run pins that the rejection is not just deleting cells at the
131161
// background lava rate: sampling the same lava field 10,000 tiles away

test/vulcanusTiles.spec.ts

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
import { describe, expect, it } from "vite-plus/test";
22

33
import fixture from "./fixtures/oracle-vulcanus-tile-names.seed123456.json";
4+
import { VULCANUS_CLIFF_BLOCKING_TILES } from "../src/noise/preview/renderVulcanusCliffs";
45
import { makeVulcanusTileResolver } from "../src/noise/tiles/vulcanusCatalog";
56

67
/**
@@ -49,4 +50,52 @@ describe("makeVulcanusTileResolver vs get_tile oracle", () => {
4950
// remaining boundary-flip count.
5051
expect(agreement).toBeGreaterThan(0.978);
5152
});
53+
54+
/**
55+
* The BINARY lava classification, separately - because that is the only thing
56+
* the cliff collision rejection reads. `tryToAddCliff` asks "does this tile
57+
* carry `water_tile`", never which tile it is, so the 19-way argmax above can
58+
* confuse `volcanic-folds` for `volcanic-folds-flat` all day without moving a
59+
* cliff. **It is exact**: measured 2026-07-30, all 49 `lava`/`lava-hot`
60+
* positions and all 332 others land on the correct side, in both directions.
61+
*
62+
* Pinned at zero rather than at a fraction. Every one of the 7 name mismatches
63+
* is a non-lava/non-lava confusion within a biome family, so nothing is being
64+
* rounded away - and the standing hypothesis this falsifies was specific: the
65+
* Vulcanus cliff rejection drops 10 real cliffs (issue #18), and
66+
* `vulcanusCliffEntities.spec.ts` recorded the guess that the resolver is
67+
* "plausibly worse at a lava boundary". It is not worse at a lava boundary;
68+
* the 42 fixture positions that sit directly on one (Chebyshev distance 1 to a
69+
* tile of the opposite lava-ness) are 42/42 correct even on the full name.
70+
*
71+
* **Know what this zero is worth before leaning on it.** Its sensitivity was
72+
* measured by planting scale factors on `lava`'s probability (2026-07-30):
73+
* `1.02` and `1.2` both still pass, `2`, `5` and `20` all fail. 381 sparse
74+
* oracle positions do not sit close enough to a lava boundary in probability
75+
* space to register a small shift, so this is a regression guard against a
76+
* broken lava range expression, NOT a sub-tile boundary check. The boundary
77+
* really is off by about a tile somewhere - the 10 dropped cliffs prove it -
78+
* and this spec cannot see that. The thing that can is the negative-space
79+
* oracle in `vulcanusCliffEntities.spec.ts`: a real cliff the game placed is a
80+
* standing assertion that the game found no lava in its box.
81+
*/
82+
it("classifies lava exactly, which is what the cliff rejection reads", () => {
83+
let ourLava = 0;
84+
let gameLava = 0;
85+
let mismatch = 0;
86+
for (let i = 0; i < positions.length; i++) {
87+
const p = positions[i];
88+
const ours = VULCANUS_CLIFF_BLOCKING_TILES.has(resolve(p.x, p.y).name);
89+
const game = VULCANUS_CLIFF_BLOCKING_TILES.has(want[i]);
90+
if (ours) ourLava++;
91+
if (game) gameLava++;
92+
if (ours !== game) mismatch++;
93+
}
94+
// Non-vacuity: the fixture really does carry lava, and plenty of it. Without
95+
// this, a resolver that returned a constant non-lava tile would pass the
96+
// mismatch assertion by agreeing with an empty set.
97+
expect(gameLava).toBe(49);
98+
expect(ourLava).toBe(49);
99+
expect(mismatch).toBe(0);
100+
});
52101
});

0 commit comments

Comments
 (0)