Skip to content

Commit 336dd2a

Browse files
authored
measure(#24): there is no ore/cliff exclusion; #18's residual is the RULE, not the fields (#57)
Verified locally and in CI, and the anti-vacuity control was tested by breaking it. ## The `+3` control is genuinely load-bearing - proven, not assumed This repo has a documented history of confirmations that turned out vacuous, so I sabotaged the substitution rather than trusting the assertion. Replacing the game-field lookup with one that never matches - i.e. simulating every lookup silently falling through to our own field: | test | with substitution dead | | --- | --- | | "substituting the game's own elevation and cliffiness does not move a single cell" | **still passes** | | "and the substitution really is live - a `+3` elevation bias does move cells" | **FAILS**: `AssertionError: expected 0 to be greater than 5` | So the headline assertion *would* have passed vacuously, and the `+3` control is exactly what catches it. Restored and re-confirmed 9/9 green. `expect(game.size).toBeGreaterThan(500)` is a second, independent guard against an empty placement set making the equality trivial. ## Provenance `test/fixtureProvenance.spec.ts` passes (4/4). Both new fixtures carry entries with `factorioVersion: "2.1.12"` and concrete `evidence` naming the capture script and recording that `refs:sync --check` reported in-sync at capture time - so neither lands as `unknown`, and the spec's `unknown` cap is not touched. ## Gate - Local `pnpm run verify` exit **0** in 60s on the exact tree that lands (`5399b6f`): format 316, lint+type 301, `check:vue` clean, app **1255 passed / 3 skipped**, worker 12. - CI `verify`: **pass** (4m41s). The branch was updated **twice** so the run tests the real target - it now contains #51, #52, #55 and #59, not the older `main` the first run saw. - No `src/` change, so nothing shipped to users; the new spec needs no Factorio at runtime because the fixtures are committed, which is why CI can run it. ## Two notes, neither blocking - The description reads as though `test/vulcanusOreCliffSeparation.spec.ts` is new. It is not - it already existed on `main` at 222 lines and this extends it to 597 (3 -> 7 `it` blocks at the top level, 9 tests total), which is why the file count stays 143 and the suite moves +4 rather than +9. - Re-deriving the binary reading and the statistics was explicitly out of scope for this pass, so the `0x1016229b4` / `0x101625038` disassembly claims and the Poisson figures are **recorded as the author's measurements, not independently reproduced here**. What was checked is that the code is gated, the fixtures have provenance, and the assertions can fail.
1 parent 0e484c0 commit 336dd2a

6 files changed

Lines changed: 95219 additions & 0 deletions

docs/noise/vulcanus-cliffs-NOTES.md

Lines changed: 176 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -522,6 +522,11 @@ the `all` path the Vulcanus DAG is evaluated four times per pixel region.
522522
> the root cause. The tables in the first two are the **pre-fix** state, kept
523523
> because the reasoning that got from one to the other is the useful part. The
524524
> current figures are in "ROOT CAUSE: `cliff_smoothing`".
525+
>
526+
> **And read "The residual is in the RULE, not the fields" at the end of this
527+
> file before spending any time on field accuracy.** Substituting the game's own
528+
> `elevation` and `cliffiness` for ours does not move a single placed cell, so
529+
> the residual this section chases is not in the fields.
525530
526531
**Superseded the "Not validated" section below.** `test/vulcanusCliffEntities.spec.ts`
527532
now compares the port against every real `cliff-vulcanus` the game places, captured
@@ -704,3 +709,174 @@ positions.
704709
That capture is the obvious next step and is the same shape as
705710
`captureCliffEntities` in `test/oracle/capture.ts`, pointed at a Vulcanus
706711
surface.
712+
713+
## The residual is in the RULE, not the fields - measured 2026-07-29
714+
715+
Everything above, and all of #18, treats the Vulcanus cliff gap as a **field
716+
accuracy** problem: the noise field matches to 5e-6, the constants are read from
717+
the game, so the ~1.15-1.19x over-placement must be residual field error
718+
somewhere. **That is wrong, and it is now measured rather than argued.**
719+
720+
`test/oracle/capture.ts vulcanus-cliff-corner-fields` dumps the game's own
721+
`vulcanus_elevation` and `cliffiness_basic` at every corner of the placement
722+
lattice (`(i*4, j*4 + 0.5)`) over three calcite-dominated 256x256 regions -
723+
12,675 corners. Feeding those values straight into our own
724+
`makeCliffPlacementFromFields` instead of our fields:
725+
726+
| region | interior cells | TP | FP | FN | precision | recall |
727+
| --- | --- | --- | --- | --- | --- | --- |
728+
| `[1500,1500]` | 3844 | 706 | 290 | 90 | 0.709 | 0.887 |
729+
| `[1100,2600]` | 3844 | 720 | 199 | 86 | 0.783 | 0.893 |
730+
| `[-1700,1900]` | 3844 | 744 | 156 | 123 | 0.827 | 0.858 |
731+
732+
and the placed cell set is **identical, cell for cell, to the one our own fields
733+
produce**. Not one cell flips in any of the three regions.
734+
735+
That is exactly what an accurate field predicts, and it is worth doing the
736+
arithmetic so the result does not read as suspicious: a cell can only flip if a
737+
corner's elevation sits within the field error of a band boundary. At ~5e-6
738+
relative error against 120-wide bands, over ~14k corner reads, the expected
739+
number of flips is ~2e-4. Zero is the expected answer.
740+
741+
So the port's Vulcanus cliff **fields are exact for placement purposes**, and
742+
17-29% of the cliff cells it places are wrong anyway, with 11-14% of the game's
743+
missed. **The residual lives in the rule** - `crossingsForChunk`'s sampling
744+
geometry, the `cliff_smoothing` knot model, `toMaybeCliffOrientation`, or
745+
`fixImpossibleCells` - and #18 should be re-pointed there.
746+
747+
`test/vulcanusOreCliffSeparation.spec.ts` pins this, including a `+3` elevation
748+
bias control so the "identical" assertion cannot pass by the substitution
749+
silently not happening (the bias moves tens of cells per region).
750+
751+
### There is no ore/cliff exclusion in the engine - read out of the binary
752+
753+
The other half of #24. Disassembled from the 2.1.12 arm64 slice (see
754+
`cliffs-NOTES.md` for the lldb recipe):
755+
756+
- `EntityMapGenerationTask::generateCliffs()` (`0x1016229b4`) calls exactly
757+
three things: `CliffGenerator::crossingsForChunk`,
758+
`CellCliffCrossing::toMaybeCliffOrientation` (inlined) and `tryToAddCliff`.
759+
**No tile lookup, no entity lookup, no resource field.**
760+
- `tryToAddCliff` (`0x101625038`) has one rejection path, `wouldCollide`, and it
761+
is gated behind `mode == 2` (`ldrb w8, [x0, #0x10]; cmp w8, #0x2; b.ne`).
762+
- `computeInternal` (`0x101622860`): `generateCliffs()`, then
763+
`generateEntities()` three times, then `generateDecoratives()`.
764+
- `apply` (`0x101623b48`): `applyCliffs()`, `applyDecoratives()`,
765+
`applyEntities()`. `applyCliffs` does call `Surface::wouldCollide` and
766+
`Entity::forceDestroy`, so a cliff can still be dropped at apply time - but
767+
against tiles and already-generated neighbours, not against ore that does not
768+
exist yet.
769+
770+
There is no separate resource generation task: `nm | c++filt` lists
771+
`BasicTilesMapGenerationTask`, `EntityMapGenerationTask` and
772+
`TileCorrectionMapGenerationTask` and nothing else of the kind, so resources are
773+
entities and they are generated after cliffs.
774+
775+
**Cliff placement is therefore a pure function of `cliff_elevation` and
776+
`cliffiness`.** There is no exclusion to port, and any "ore excluded from cliffs"
777+
item (the M3a follow-up in `client-preview-ROADMAP.md`, #24's own second cause)
778+
should be closed rather than implemented.
779+
780+
Collision was already ruled out; both masks were re-read here rather than quoted
781+
from the issue - cliff `{item, meltable, object, player, water_tile,
782+
is_lower_object, is_object, cliff}`, resource `{resource}`. **Tiles cannot
783+
separate them either**, which is worth recording because the ore patches really
784+
do paint their own tile: `volcanic-jagged-ground`'s autoplace is
785+
`5 * min(10, max(vulcanus_calcite_region + 0.2, ...))`, and
786+
`tiles-vulcanus.lua` labels it "CLIFF TILE". Its mask is
787+
`tile_collision_masks.ground()` = `{ground_tile}`, which the cliff mask does not
788+
touch. Of the ~20 Vulcanus tiles only `lava` and `lava-hot` carry `water_tile`,
789+
and `tile_collision_masks.lava()` also carries `resource`, so lava excludes both.
790+
791+
### #24's "100x below chance" baseline does not hold
792+
793+
The issue divides the observed overlap by a **tile-independence** baseline
794+
(`ore tiles x cliff coverage / area`), which assumes each ore tile is an
795+
independent trial. It is not: region `[0,0]`'s 945 ore tiles are **2 connected
796+
blobs**, and `[-1200,800]`'s 1047 are **2**.
797+
798+
The right null keeps the blobs and moves them - shift the whole ore tile set on
799+
the region torus and re-measure. 500 shifts per region, over the three committed
800+
regions plus six new ones (`test/oracle/capture.ts
801+
vulcanus-ore-cliff-replication`, region list fixed before any was measured, and
802+
the two that turned out to hold no ore are kept in the fixture rather than
803+
dropped):
804+
805+
| region | ore tiles | blobs | cliff cover | overlap | tile-indep. | shift median | P(shift <= obs) |
806+
| --- | --- | --- | --- | --- | --- | --- | --- |
807+
| `[0,0]` | 945 | 2 | 6.9% | 0 | 65 | 0 | **0.51** |
808+
| `[1500,1500]` | 3933 | 25 | 21.6% | 8 | 850 | 789 | **0.000** |
809+
| `[-1200,800]` | 1047 | 2 | 9.8% | 0 | 103 | 49 | **0.29** |
810+
| `[700,-1800]` | 404 | 9 | 15.2% | 0 | 61 | 55 | 0.02 |
811+
| `[-2400,-600]` | 597 | 3 | 12.9% | 1 | 77 | 58 | 0.19 |
812+
| `[1100,2600]` | 3045 | 20 | 21.2% | 9 | 645 | 611 | **0.000** |
813+
| `[-900,-2500]` | 904 | 4 | 15.9% | 0 | 144 | 110 | 0.18 |
814+
| `[-1700,1900]` | 714 | 2 | 22.7% | 0 | 162 | 156 | 0.10 |
815+
| `[300,3400]` | 944 | 8 | 2.0% | 0 | 19 | 0 | 0.73 |
816+
817+
**The separation replicates** - 18 of 12,533 ore tiles pooled, 0.14%, with both
818+
ore-rich regions outside 500 of 500 shifts. **But `[0,0]` and `[-1200,800]` were
819+
never evidence for it**: half of all random placements of `[0,0]`'s blob also hit
820+
zero cliffs. The previous write-up read their "ratio to chance 0.000" as the
821+
strongest signal in the set; it is the weakest, and the "no cliff can exist below
822+
70, so region 0 is explained by elevation" reading was explaining a
823+
non-observation.
824+
825+
### What is left open, stated as a contradiction rather than a cause
826+
827+
Put the two halves together and they do not fit:
828+
829+
- the binary says cliff placement cannot see ore; and
830+
- the game's own `elevation` + `cliffiness`, through our rule, place **47** cliff
831+
cells inside cells whose full 4x4 footprint is ore (172/130/29 such cells per
832+
region, 8/38/1 placed) where the game placed **0**.
833+
834+
A covariate-matched control says that is not the rule simply being worse on
835+
volcano terrain: pairing each full-ore cell with up to three no-ore cells at the
836+
same mean elevation (+/-25) and mean cliffiness (+/-0.1) in the same region, the
837+
rule's precision on the controls is 0.79 / 1.02 / 2.00 (n = 443/387/69). So the
838+
47 should have been ~47 real cliffs. Poisson P(0 | 47) ~ 4e-21.
839+
840+
Since the binary reading is direct, the modelling of `crossingsForChunk` is the
841+
suspect, and **a 4x4 cell fully inside a calcite patch is the sharpest test case
842+
anyone has for #18**: our rule is wrong there ~100% of the time while being right
843+
~78% of the time everywhere else. Four explanations for the localisation were
844+
measured and **falsified** on 2026-07-29 - do not re-test them:
845+
846+
1. **Elevation.** With the cliffiness gate forced open, the game's own corner
847+
elevations give a band crossing in 40.5% / 34.3% / 50.0% of full-ore cells
848+
against 42.7% / 37.3% / 34.0% of random cells in the same regions. Calcite
849+
sits on band-crossing terrain at the background rate. (For coal and tungsten
850+
the figure is 0.000 in four of five regions, and that is a real structural
851+
result rather than a coincidence: `vulcanus_ashlands_func` is
852+
`300 + 0.001 * min(basis, basis)`, i.e. flat 300, and the basalts branch tops
853+
out near 120 against `cliff_elevation_0 = 70`. Coal and tungsten live on
854+
terrain that cannot host a cliff at all. Calcite and the geysers live in the
855+
mountains, with the cliffs, and are the only hard case.)
856+
2. **Cliffiness.** The gate is `cliffinessAvg > 0.5` and `cliffiness_basic`
857+
floors at exactly 0.5, so it is a hard binary gate over roughly half the map,
858+
not a soft one. It is open at 14.9% of `[1500,1500]`'s full-ore cells against
859+
60.0% of random - but at **85.8%** of `[1100,2600]`'s against 64.0%, and 10.0%
860+
at `[-1700,1900]`. It does not replicate in either direction, which is what a
861+
coincidence looks like: `cliffiness_basic` is a `quick_multioctave_noise` at
862+
`seed1 = 123`, `input_scale = 1/32`, with no dependence on any resource,
863+
biome or elevation field, so there is no path by which ore could correlate
864+
with it. **This one is worth naming as a near-miss** - in region 1 alone it
865+
looked like a 3.8x mechanism, and one region would have been enough to write
866+
it down as the answer.
867+
3. **`fixImpossibleCells`.** On and off changes the full-ore predictions by 0
868+
(8/35/1 both ways on the interior-inset window).
869+
4. **Steep or aliased terrain.** Full-ore cells' max corner-to-corner elevation
870+
delta is p10/p50/p90 = 17/37/63 against 13/35/66 for no-ore cells - the same
871+
distribution - and the rule's precision is 0.58-0.84 across every delta bin,
872+
with no bin where it collapses. (This was the best remaining guess: the
873+
mountains branch carries `200 * (aux - 0.5) * (mountain_volcano_spots + 0.5)`
874+
and `vulcanus_aux` is a 5-tile-wavelength noise, so elevation there swings
875+
~150 between adjacent 4-tile corners. It swings the same amount off the ore.)
876+
877+
One measured fact that is not a cause but is where a fifth hypothesis should
878+
start: the false positives our rule produces **off** ore sit one cell from a real
879+
cliff 78% of the time (93 of 120 at `[1100,2600]`) - they are edge-of-line
880+
offsets along a real cliff face. The ones **inside** ore sit 2-5 cells away (4 of
881+
37 at distance 1). They are not misaligned cliff lines; they are cliff faces the
882+
game does not have at all.

test/fixtures/PROVENANCE.json

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -184,6 +184,10 @@
184184
"factorioVersion": "2.1.12",
185185
"evidence": "stated in the fixture's own _comment"
186186
},
187+
"oracle-vulcanus-cliff-corner-fields.seed123456.json": {
188+
"factorioVersion": "2.1.12",
189+
"evidence": "captured 2026-07-29 by test/oracle/capture.ts vulcanus-cliff-corner-fields against the installed binary, which pnpm refs:sync --check reported in sync at 2.1.12 at capture time"
190+
},
187191
"oracle-vulcanus-cliff-entities.seed123456.json": {
188192
"factorioVersion": "2.1.12",
189193
"evidence": "captured 2026-07-28 by test/oracle/capture.ts vulcanus-cliff-entities against the installed binary, which pnpm refs:sync --check reported in sync at 2.1.12 at capture time"
@@ -208,6 +212,10 @@
208212
"factorioVersion": "2.1.12",
209213
"evidence": "stated in the fixture's own _comment"
210214
},
215+
"oracle-vulcanus-ore-cliff-replication.seed123456.json": {
216+
"factorioVersion": "2.1.12",
217+
"evidence": "captured 2026-07-29 by test/oracle/capture.ts vulcanus-ore-cliff-replication against the installed binary, which pnpm refs:sync --check reported in sync at 2.1.12 at capture time"
218+
},
211219
"oracle-vulcanus-resource-entities.seed123456.json": {
212220
"factorioVersion": "2.1.12",
213221
"evidence": "captured 2026-07-28 by test/oracle/capture.ts vulcanus-resource-entities against the installed binary, which pnpm refs:sync --check reported in sync at 2.1.12 at capture time"

0 commit comments

Comments
 (0)