Record ECP-7 Batch 18 attenuated replay result

README.md

@@ -26,16 +26,18 @@ The protocol uses no words or alphabet, but four meaning-free local symbols. For
 See [`docs/results-ecp6.md`](docs/results-ecp6.md) for the conclusion, [`docs/protocol-specification-ecp6.md`](docs/protocol-specification-ecp6.md) for the wire format and [`evidence/ecp6/report.md`](evidence/ecp6/report.md) for the compact confirmatory evidence.
 
 ECP-7 development now tests how much of that result depends on the explicit
-one-factor-per-slot architecture. Its first seventeen sealed batches are valid
+one-factor-per-slot architecture. Its first eighteen sealed batches are valid
 development results but none passes the full gate. Batch 15 remains strongest:
 the position-aware design at 30,000 optimization steps reaches 83.46% train
 exactness, 82.59% validation and 83.37% translator validation while using
 12,585–13,200 hard messages per sender. Validation and translator thresholds pass
 together, but the registered train and injectivity thresholds do not. Batch 16
 added late worst-factor pressure and regressed to 76.46% validation. Batch 17
 replayed globally mined training collisions and modestly improved code use, but
-still regressed to 77.09% validation. The ECP-6 positive controls remain perfect
-and the ECP-7 confirmatory test remains sealed. See
+still regressed to 77.09% validation. Batch 18 reduced replay to task-loss scale,
+recovering 80.71% validation and a new-best 84.06% translator score, but train
+exactness and injectivity still failed. The ECP-6 positive controls remain
+perfect and the ECP-7 confirmatory test remains sealed. See
 [`docs/development-log-ecp7.md`](docs/development-log-ecp7.md).
 
 ## Build on this work
@@ -93,6 +95,7 @@ Do not translate those artifacts in place. New experiment configurations and all
 - [`config/ecp7-b15-development.yaml`](config/ecp7-b15-development.yaml) — the Batch 10 design with a 30,000-step population horizon.
 - [`config/ecp7-b16-development.yaml`](config/ecp7-b16-development.yaml) — late normalized factor-minimax pressure on the Batch 15 base.
 - [`config/ecp7-b17-development.yaml`](config/ecp7-b17-development.yaml) — late training-only global collision-pair replay on the Batch 15 base.
+- [`config/ecp7-b18-development.yaml`](config/ecp7-b18-development.yaml) — the Batch 17 replay mechanism with final weight reduced to 0.1.
 - [`config/ecp7-positive-control-development.yaml`](config/ecp7-positive-control-development.yaml) — ECP-6 positive control on the ECP-7 split.
 - [`config/ecp0.yaml`](config/ecp0.yaml) — machine-readable configuration from step 1.
 - [`config/ecp1.yaml`](config/ecp1.yaml) — frozen configuration of the population trial.

config/ecp7-b18-development.yaml

@@ -0,0 +1,13 @@
+extends: ecp7-b17-development.yaml
+
+experiment:
+  title: ECP-7 attenuated late global collision-pair replay
+  arm: attenuated_late_global_collision_pair_replay
+  objective: >-
+    Test whether keeping the complete Batch 17 collision-replay mechanism but
+    limiting its final coefficient to task-loss scale can improve injectivity
+    without disrupting the Batch 15 semantic trajectory.
+
+training:
+  global_collision_replay:
+    weight: 0.1

docs/AI_AGENT_START.md

@@ -41,7 +41,7 @@ python3.12 -m venv .venv
 .venv/bin/ecp6 --config config/ecp6.yaml validate
 ```
 
-Expected baseline: 100 passing tests and split sizes `14336/1024/1024`.
+Expected baseline: 102 passing tests and split sizes `14336/1024/1024`.
 
 ## 4. Reproduce ECP-6
 
@@ -71,7 +71,7 @@ This reruns a known experiment. It must not be presented as a new independent co
 
 ## 5. Continue ECP-7 scientifically
 
-Do not modify ECP-0 through ECP-6 or the completed ECP-7 Batch 1 through Batch 17
+Do not modify ECP-0 through ECP-6 or the completed ECP-7 Batch 1 through Batch 18
 configs. ECP7-B1-I collapsed to 130–139 hard messages. ECP7-B2-I improved its
 soft objective but collapsed further to 85–104 hard messages. ECP7-B3-I applied
 that loss to straight-through hard messages and improved to 585–972 messages,
@@ -110,11 +110,15 @@ validation and 77.76% translator validation without solving injectivity. Batch
 It modestly improved unique-message use but regressed to 83.15% train, 77.09%
 validation and 83.01% translator validation without becoming injective. The
 confirmatory ECP-7 test is still sealed.
+ECP7-B18-I reduced replay weight to `0.1` and recovered 84.08% train, 80.71%
+validation and a new-best 84.06% translator validation. Validation and
+translator gates pass, but train and injectivity do not. Batch 15 retains the
+best validation result, and the confirmatory ECP-7 test remains sealed.
 
 Continue with this sequence:
 
 1. Read `docs/research-design-ecp7.md` and `docs/development-log-ecp7.md`.
-2. Define exactly one ECP7-B18 intervention and its failure criterion.
+2. Define exactly one ECP7-B19 intervention and its failure criterion.
 3. Register the variant and immutable configuration hashes before training.
 4. Add tests for every new invariant.
 5. Use only `smoke` and `develop`; keep the ECP-7 test split sealed.
@@ -127,25 +131,25 @@ Continue with this sequence:
 
 ## 6. Recommended next experiment
 
-Batch 15 remains the strongest weak-structure result. Batch 17 shows that global
-collision replay provides a usable signal and modestly improves code use, but
-its full weight overwhelms the remaining task objective and causes collision
-churn. The clean next question is whether the same mechanism works at a
-task-scale coefficient. A clean ECP7-B18 progression is:
+Batch 18 shows that replay at task-loss scale avoids B17's early collapse and
+improves train and translator scores, but validation peaks at step 22,400 and
+then declines while replay remains active. The clean next question is whether a
+bounded replay pulse can preserve early collision reduction and then return to
+unperturbed semantic refinement. A clean ECP7-B19 progression is:
 
 - keep the same world and bit budget;
 - keep the complete B15 architecture, horizon and base losses;
 - keep the complete B17 training-only collision mining, replay batch,
   temperature and refresh mechanism;
 - keep replay weight `0` through step 15,000, warm only to `0.1` at step 20,000,
-  then hold;
+  then decay linearly to `0` at step 25,000 and hold;
 - measure injectivity, validation composition and new-reader induction;
 - rerun the ECP-6 architecture as the frozen positive control.
 
 Continue to use training meanings only for pair mining. Do not change the
 mining source, pair sampler, architecture, base utilization, optimizer, data,
 temperature, duration, validation cadence or development thresholds, and do
-not combine the coefficient change with factor reweighting, variable-length
+not combine the schedule change with factor reweighting, variable-length
 messages or negotiation.
 
 ## 7. Definition of done for any contribution

docs/development-log-ecp7.md

@@ -25,6 +25,7 @@ this log.
 | ECP7-B15-I | Intervention | B10 with 30,000-step optimization | Position-aware MLP | completed; gate failed |
 | ECP7-B16-I | Intervention | B15 with late normalized factor minimax | Position-aware MLP | completed; gate failed |
 | ECP7-B17-I | Intervention | B15 with late global collision-pair replay | Position-aware MLP | completed; gate failed |
+| ECP7-B18-I | Intervention | B17 replay with final weight 0.1 | Position-aware MLP | completed; gate failed |
 
 ## Batch 1 preregistration
 
@@ -1258,3 +1259,97 @@ step 20,000 and then hold. This makes an initially near-one replay loss
 comparable to the remaining task loss instead of several times larger. No
 mining, pair batch, refresh, architecture, base loss, duration, data, translator
 or threshold change may accompany that coefficient test.
+
+## Batch 18 preregistration
+
+Preregistered: July 18, 2026 at 14:53:31 UTC<br>
+Seed: `11`<br>
+Maximum population steps: `30,000`<br>
+Minimum selection steps: `15,000`<br>
+Early-stopping patience: `5,000` steps<br>
+Replay start: step `15,000`<br>
+Replay weight: `0` through step `15,000`, linear to `0.1` at step `20,000`, then hold<br>
+Replay pairs per sender and update: `32`<br>
+Training-codebook refresh interval: `200` steps<br>
+Relaxed replay temperature: `1.0`<br>
+Raw configuration SHA-256: `a9da48a2c76fd01a9f2889050c347ed9fcf616389297424f156e9f2d8489e59f`<br>
+Effective configuration SHA-256: `21d76380640738ee98ddbdb7ae88c6a1d3185add860b96f1fe9c5c83a782d757`<br>
+Split-SHA-256: `4947058c75ab07cb43a87eb82776b12cb2a7e2eeba7114de110d3b852cbc64cd`<br>
+Test unsealed: **no**
+
+ECP7-B18-I keeps the entire Batch 17 mechanism and every Batch 15 model,
+training and evaluation setting. It changes only the replay coefficient from
+`1.0` to `0.1`, preserving the same start and 5,000-step linear warmup. This
+tests whether replay can retain its code-use improvement when its weighted
+magnitude remains comparable to the task loss. The unchanged positive control
+is rerun, and no alternative coefficient or accompanying intervention is
+admitted into this batch.
+
+## Batch 18 results
+
+Positive-control run: `runs/ecp7-batch18-control-development/20260718T145534Z-ecp7-development`<br>
+Intervention run: `runs/ecp7-batch18-intervention-development/20260718T145534Z-ecp7-development`<br>
+Test unsealed: **no**
+
+| Metric | Positive control | ECP7-B18-I |
+|---|---:|---:|
+| Population train, mean | 100% | **84.0777%** |
+| Population train, worst link | 100% | 82.1429% |
+| Population validation, mean | 100% | **80.7129%** |
+| Population validation, worst link | 100% | 78.6133% |
+| Universal translator, validation | 100% | **84.0576%** |
+| Exact sender-message agreement | 100% | 90.13% |
+| Unique messages per sender | 15,360 | 12,720–13,442 |
+| Collision meanings per sender | 0 | 1,918–2,640 |
+| Message entropy | 13.91 bits | 13.55–13.64 bits |
+| Development gate | pass | **fail** |
+
+The positive control passed every gate at 100%. ECP7-B18-I passed validation
+and universal-translator thresholds together. It set the strongest ECP-7 train
+and translator scores so far, but still failed both known-meaning train
+thresholds and sender injectivity. The joint gate therefore fails and
+confirmatory access remains unauthorized.
+
+The trajectory through step 15,000 again matches B15 exactly. At step 18,000,
+replay weight was `0.06`, train exactness was 83.61% and validation was 79.52%.
+At step 20,000 the coefficient reached its registered maximum `0.1` and
+validation crossed 80%.
+
+The selected checkpoint was step 22,400. Replay loss was `0.80354`, so its
+weighted contribution was approximately `0.0804`, close to task loss `0.06669`
+and far below B17's selected `0.201` contribution. Utilization loss was
+`-0.75717`. The mined collision-pair banks contained 2,688, 2,240, 2,777 and
+2,135 pairs.
+
+The lower coefficient prevented B17's early collapse but did not make replayed
+pairs stably separable: replay loss returned near one at several later
+checkpoints while pair banks continued changing. Validation declined after the
+selected checkpoint. The registered patience rule stopped at step 27,400, where
+train exactness had risen to 84.49% but validation had fallen to 78.97%.
+
+Compared with B15, three senders had fewer collision meanings and the fourth
+was unchanged, but none became injective. Sender agreement remained below B15
+at 90.13%. Population validation factor accuracies were
+`[95.97%, 99.77%, 98.44%, 85.59%]`; universal-translator factor accuracies were
+`[100%, 100%, 98.44%, 85.62%]`. The replay tradeoff again slightly improved
+texture and code use while weakening population color generalization.
+
+Both sealed analyses report 65 matching artifact hashes, 16,384 validation-only
+episodes, no confirmatory-test keys, valid local alphabets, and exactly 14
+declared bits for every logged message.
+
+## Batch 18 decision
+
+ECP7-B18-I is a valid partial improvement but remains negative under the joint
+gate. Coefficient attenuation recovers validation and creates new-best train
+and translator scores, confirming that B17's principal failure was objective
+scale. It still does not approach the train thresholds or injectivity, and
+Batch 15 retains the highest validation score. The confirmatory split remains
+sealed.
+
+A future ECP7-B19 may keep B18 exactly through step 20,000, then decay only the
+replay coefficient linearly from `0.1` to `0` over steps 20,000–25,000 and hold
+it at zero thereafter. This tests whether a bounded replay pulse can preserve
+the early collision reduction while returning the final phase to pure B15 task
+and utilization refinement. Mining, pair batch, refresh, architecture, base
+losses, optimizer, duration, data, translator and thresholds must not change.

docs/research-design-ecp7.md

@@ -755,3 +755,44 @@ sealed-test requirement fails. No alternative mining source, refresh interval,
 pair batch size, temperature, weight, warmup, architecture, duration, optimizer,
 data, translator or seed may be tried inside Batch 17. Failure is recorded
 without confirmatory test access.
+
+## Batch 18 preregistration — Attenuated collision replay
+
+Status: **preregistered for sealed development**<br>
+Preregistered: July 18, 2026 at 14:53:31 UTC<br>
+Configuration: `config/ecp7-b18-development.yaml`<br>
+Raw configuration SHA-256: `a9da48a2c76fd01a9f2889050c347ed9fcf616389297424f156e9f2d8489e59f`<br>
+Effective configuration SHA-256: `21d76380640738ee98ddbdb7ae88c6a1d3185add860b96f1fe9c5c83a782d757`
+
+Batch 17 established that global training-codebook mining produces a usable
+collision signal and modestly improves hard code use. Its replay weight rose to
+`0.24` at the selected checkpoint, where replay loss `0.83844` contributed
+approximately `0.201` to the objective versus task loss `0.07453`. Higher
+weights then caused collision churn, validation regression and early stopping.
+
+Batch 18 tests only whether coefficient scale caused that failure. ECP7-B18-I
+inherits the complete B17 mining algorithm, training-only pair source,
+seed-derived sampler, 32-pair sender batches, 200-step refresh interval, relaxed
+temperature `1.0`, architecture, base losses, optimizer, data, 30,000-step
+duration, selection window, translator and thresholds. The sole change is:
+
+- replay weight remains `0` through step `15,000`;
+- replay weight rises linearly to `0.1` over steps `15,000–20,000`;
+- replay weight remains `0.1` through step `30,000`.
+
+For an initially near-one replay loss, full weight `0.1` is comparable to the
+remaining B15 task-loss scale instead of several times larger. The task-batch
+trajectory and all random sampling through step 15,000 must again match B15
+exactly.
+
+The batch contains exactly two seed-11 runs:
+
+1. the unchanged ECP-6 positive control;
+2. ECP7-B18-I with attenuated late global collision-pair replay.
+
+All existing validity and development gates remain unchanged. The intervention
+fails if any train, validation, translator, injectivity, channel-integrity or
+sealed-test requirement fails. No alternative coefficient, schedule, mining
+source, pair sampler, refresh, temperature, architecture, duration, optimizer,
+data, translator or seed may be tried inside Batch 18. Failure is recorded
+without confirmatory test access.

evidence/ecp7-development/README.md

@@ -93,6 +93,11 @@ It improved hard code use modestly but regressed to 83.15% train exactness,
 77.09% validation and 83.01% translator validation without reaching
 injectivity. Exact identities and hashes are in `batch17-manifest.json`.
 
+Batch 18 reduced final replay weight to 0.1. It recovered 84.08% train
+exactness, 80.71% validation and a new-best 84.06% translator validation, but
+still failed train thresholds and injectivity. Exact identities and hashes are
+in `batch18-manifest.json`.
+
 Reproduce the development arms with:
 
 ```bash

evidence/ecp7-development/batch18-manifest.json

@@ -0,0 +1,49 @@
+{
+  "batch": "ECP7-B18",
+  "confirmatory_test_unsealed": false,
+  "config_effective_sha256": "21d76380640738ee98ddbdb7ae88c6a1d3185add860b96f1fe9c5c83a782d757",
+  "config_raw_sha256": "a9da48a2c76fd01a9f2889050c347ed9fcf616389297424f156e9f2d8489e59f",
+  "control": {
+    "artifact_hashes": {
+      "effective_config.yaml": "4bfb976293f117d3ae35f790e4fd6371628be95d91febb334bc275dc5d6b66d4",
+      "posthoc-analysis.json": "ee9087f926799bdabed71a7402b1638287f32e1854cf478911d180e6c6c18f9a",
+      "seed-11/metrics.json": "1970894d5812b34d291d4bf072787e03f00acb881095c67152d7cc413ab5d227",
+      "summary.json": "16829b4b3053489081bbc9aea3d6343184386f33da3691820bf7a6528a8bb7b2"
+    },
+    "development_gate_passed": true,
+    "population_train_mean_exact_match": 1.0,
+    "population_validation_mean_exact_match": 1.0,
+    "run_id": "20260718T145534Z-ecp7-development",
+    "universal_translator_validation_exact_match": 1.0,
+    "unique_messages_per_sender": [15360, 15360, 15360, 15360]
+  },
+  "intervention": {
+    "artifact_hashes": {
+      "effective_config.yaml": "a54386103f91d18ca9832aa3bfaa31f8b01c7adf3c52a32502ba5bcac628ebc0",
+      "posthoc-analysis.json": "2d14b9a84b3cada9b63a0eecfc211636e6af59779b0f29d515995b1e43654dfb",
+      "seed-11/metrics.json": "6ba9ee04b39403db8b8f11291f61f1a806d2a5165a5e1a2243117bade4178c94",
+      "summary.json": "861f73b41423dbc74781a4db2bab4232e078541b49f73059dd796c336273ba03"
+    },
+    "best_code_utilization_loss": -0.7571721076965332,
+    "best_global_collision_replay_loss": 0.8035427927970886,
+    "best_global_collision_replay_pair_counts": [2688, 2240, 2777, 2135],
+    "best_global_collision_replay_weight": 0.1,
+    "best_step": 22400,
+    "collision_meanings_per_sender": [2520, 2400, 2640, 1918],
+    "development_gate_passed": false,
+    "exact_sender_message_agreement": 0.9012912326388888,
+    "final_step": 27400,
+    "message_entropy_bits_per_sender": [13.56107291977656, 13.594194010321495, 13.546725724142203, 13.638984103857606],
+    "population_train_mean_exact_match": 0.8407767159598214,
+    "population_train_worst_link_exact_match": 0.8214285714285714,
+    "population_validation_factor_accuracy": [0.95965576171875, 0.99774169921875, 0.984375, 0.85589599609375],
+    "population_validation_mean_exact_match": 0.80712890625,
+    "population_validation_worst_link_exact_match": 0.7861328125,
+    "run_id": "20260718T145534Z-ecp7-development",
+    "universal_translator_validation_exact_match": 0.840576171875,
+    "unique_messages_per_sender": [12840, 12960, 12720, 13442]
+  },
+  "seed": 11,
+  "split_artifact_sha256": "676195997ee0c77d5c9565bae045dbb938b432a2e4e74b666233b0a41f924718",
+  "split_sha256": "4947058c75ab07cb43a87eb82776b12cb2a7e2eeba7114de110d3b852cbc64cd"
+}

tests/conftest.py

@@ -128,3 +128,8 @@ def ecp7_b16_config():
 @pytest.fixture(scope="session")
 def ecp7_b17_config():
     return load_config(PROJECT_ROOT / "config" / "ecp7-b17-development.yaml")
+
+
+@pytest.fixture(scope="session")
+def ecp7_b18_config():
+    return load_config(PROJECT_ROOT / "config" / "ecp7-b18-development.yaml")

tests/test_config.py

@@ -615,3 +615,17 @@ def test_global_collision_replay_parameters_are_bounded(ecp7_b17_config):
     ] = 0
     with pytest.raises(ConfigError, match="temperature must be positive"):
         validate_config(invalid_temperature)
+
+
+def test_ecp7_b18_changes_only_the_collision_replay_weight(
+    ecp7_b17_config, ecp7_b18_config
+):
+    assert ecp7_b18_config["world"] == ecp7_b17_config["world"]
+    assert ecp7_b18_config["dataset"] == ecp7_b17_config["dataset"]
+    assert ecp7_b18_config["channel"] == ecp7_b17_config["channel"]
+    assert ecp7_b18_config["agents"] == ecp7_b17_config["agents"]
+    b17_training = deepcopy(ecp7_b17_config["training"])
+    b18_training = deepcopy(ecp7_b18_config["training"])
+    assert b17_training["global_collision_replay"].pop("weight") == 1.0
+    assert b18_training["global_collision_replay"].pop("weight") == 0.1
+    assert b18_training == b17_training