This is exploration-phase raw material, honestly labelled — not three finished results.
Three times in June 2026 I proposed a result about how neural networks generalize. Three times I built the control that killed it. What is here is the wreckage plus the raw numbers, and one command that re-derives every published figure from them.
make
≈10 seconds. Python 3.9+, standard library only. No GPU, no network, no downloads, no model
weights, no pip install. The inputs are 27 KB of JSON committed in evidence/.
It runs 40 checks, prints what each one can and cannot certify, and exits non-zero if any fails.
Give a robot a structured belief over its own state. If it knows what it doesn't know, that ought to port to a different body. Leave-one-out over 10 Open-X embodiments says it does not.
3 win · 3 lose significantly · 4 undecided. At the embodiment level — the unit you are generalizing across — the mean is -0.071, 95% CI [-0.283, +0.142], which includes zero.
This is the strongest of the three, for reasons that were right in the lab rather than in the packaging: the source project shipped a shuffled-label null control beside every real number (0.503–0.517, i.e. chance), used the right sampling unit, and the result survived its strongest objection — under the sign-agnostic statistic |AUROC−0.5| the interval still includes zero, mean +0.009, CI [-0.055, +0.073].
Two things must travel with it. DEFECTS.md D-14: the same source project contains a
mock evaluation harness whose conclusions were retracted, and you cannot verify from inside
this package that these files are not from it — a reader forged the provenance file and the check
passed. D-13: the two shipped runs disagree by up to 0.257 AUROC, and the headline split depends
on which one you read.
Ablate each channel, see which ones drop the correct-class logit, and you get a causal attribution vector per image. Same-class images ought to share one — a generalizing net has stable causal coalitions, a memorizing net does not.
It is intrinsic dimension wearing a costume. partial(specificity, dimension | test) = −0.94,
and at matched dimension a noise-memorizing net wins on it.
The surviving estimate has moved four times, most recently after I declared it settled. −0.94 → −0.978 → −0.641 → −0.304 under a three-level control, because the design has three arms and I coded it as two. The residual does not clear a permutation null (p ≈ 0.072) — but two of three strata support it (+0.81, +0.84), and I published every number that damaged it for eight rounds while publishing none of these. Quote the range and the confound, never a single number.
The project's own figure says "robot-state 66% BEATS the best leaderboard model by 2×." Its own later, wider sweep says 54% and 48%.
Tracing the baselines — hardcoded in four files, cited nowhere — to arXiv:2510.13626v3 shows the comparison fails three ways: the model labelled "best" is the paper's own model, fine-tuned on 20,000 target-distribution trajectories; five models in that same table beat the 30.3 the figure calls the maximum, which the paper itself bolds as highest — an error in a published paper; and the Total column is episode-weighted while mine was an unweighted mean.
Honest result: ranks 6–8 of 12, tied with π0 (56.13 against 56.00 like-for-like). This section is the only place this package tells you something you did not already know.
| file | what it is |
|---|---|
DEFECTS.md |
Twenty-eight defects, three-valued. Seventeen are mine, and six of those I introduced while fixing what readers had just shown me. Start at the "if you read only three" box |
RESULTS.md |
Every number, generated by the handle. Never hand-edited |
LIMITS.md |
What this does not claim, in my own voice |
COLD_READ.md |
What fourteen independent readers found across nine rounds |
evidence/ |
11 files, 27 KB. Everything above is computed from these |
verify/ |
The handle |
It certifies arithmetic and reasoning from the recorded measurements. Independent readers
re-derived all three headlines from evidence/ with their own code, several matching to 1e-16.
It does not certify that the measurements were taken correctly. Regenerating them needs an RTX 5080, a 2.9 GiB dataset cache, a 21 GB checkpoint and a physics simulator. I re-derived every statistic; I did not re-run a single experiment. Nobody has, including me.
Verdicts are three-valued throughout — CONFIRMED · OVERTURNED · UNVERIFIED — and UNVERIFIED never collapses into OVERTURNED. A false acquittal is permanent, because nobody re-examines a cleared claim.
The verification apparatus is now its own library:
tool.research-verification.proxy-ledger-harness.reusable-library… — standard library only, with
its own worked example and its own LIMITS. Three of fourteen readers named it independently as the
thing worth taking, and one called it "a paper-sized contribution currently packaged as an
appendix to three dead results."
This line used to say 1,004 lines, which was exact when written and is now wrong by a third — the library has since gained three-valued verdicts, a build-residue gate and a CLI. The count is gone rather than corrected, because nothing on either side can hold it: the pin map guards a document against its own
derive(), and this package has noderive()that counts another package's source. A number you cite about somebody else's tree is outside every guarantee here.find . -name '*.py' | xargs wc -lin that directory gives today's figure.
Its honest ceiling is documented there and applies here: it catches drift, not error. Three times in this package a full green coexisted with a defect a reader found in minutes.
make falsify
plants one violation per check and asserts each fires — and refuses to credit a check that was already red before anything was planted.
The programme these were meant to become is Embodied Generalization Migration. This is not that. EGM's §3.2 names P7 and P8 as its prehistory — "early invariant projects: the results were usually retractions, not confirmations" — and its method comes from two language-model experiments that are not in this package. What is here is that prehistory, plus a forensic slice of EGM's exemplar.
MIT for this package. CIFAR-10, Open-X Embodiment, LIBERO-plus and the upstream
Mondo-Robotics/DiT4DiT fork are referenced, not redistributed.