Add ECP-8 Batch 1 surplus-capacity study

AGENTS.md

@@ -41,6 +41,6 @@ python3.12 -m venv .venv
 - Keep existing ECP-0 through ECP-6 configurations reproducible.
 - Document new experiments in three stages: preregistration, sealed development log, confirmatory results.
 - Write new configurations, documentation, reports and user-facing output in English.
-- When proposing ECP-7, prefer a single falsifiable intervention over several simultaneous architecture changes.
+- Preserve the completed ECP-8 Batch 1 capacity isolation and its sealed confirmatory split. Any Batch 2 must keep the 16-bit arm fixed and preregister exactly one structure-induction mechanism.
 
-The recommended next research question is whether compositional structure still emerges when the explicit one-factor-per-slot architectural bias is weakened.
+ECP-8 Batch 1 found that surplus capacity nearly removes collisions but does not reach injectivity or compositional validation. Do not continue a channel-size search.

README.md

@@ -73,6 +73,16 @@ configuration was frozen and its confirmatory split was never opened. See
 The compact synthesis is in
 [`docs/ecp7-development-conclusion.md`](docs/ecp7-development-conclusion.md).
 
+ECP-8 Batch 1 is a valid negative sealed-development result. It tested whether
+the strongest generic ECP-7 architecture becomes injective when all four
+positions may use the complete 16-symbol vocabulary: 16 fixed bits instead of
+the 14-bit lower bound. Compared with the paired 14-bit control, mean train
+exactness rose from 79.55% to 98.76%, worst-link validation from 64.84% to
+73.34%, and the minimum codebook from 12,048 to 15,095 messages. The arm still
+had 225–265 collisions per sender and failed both 80% validation thresholds.
+The ECP-8 confirmatory split remains sealed. See
+[`docs/development-log-ecp8.md`](docs/development-log-ecp8.md).
+
 ## Build on this work
 
 A new developer or fresh AI agent can start with no previous chat context. First read [`AGENTS.md`](AGENTS.md) for the binding research rules and then follow [`docs/AI_AGENT_START.md`](docs/AI_AGENT_START.md) for installation, architecture overview, reproduction and proper setup of ECP-7.
@@ -112,6 +122,8 @@ Do not translate those artifacts in place. New experiment configurations and all
 - [`docs/research-design-ecp7.md`](docs/research-design-ecp7.md) — preregistered weak-structure question, paired control and sealed split.
 - [`docs/development-log-ecp7.md`](docs/development-log-ecp7.md) — the first negative weak-structure batch and integrity checks.
 - [`docs/ecp7-development-conclusion.md`](docs/ecp7-development-conclusion.md) — sealed development synthesis and closure after 29 variants.
+- [`docs/research-design-ecp8.md`](docs/research-design-ecp8.md) — preregistered surplus-capacity question, fresh split and three-arm gate.
+- [`docs/development-log-ecp8.md`](docs/development-log-ecp8.md) — sealed ECP-8 development record.
 - [`config/ecp7-development.yaml`](config/ecp7-development.yaml) — sealed weak-structure intervention configuration.
 - [`config/ecp7-b2-development.yaml`](config/ecp7-b2-development.yaml) — factor-agnostic soft code-utilization intervention.
 - [`config/ecp7-b3-development.yaml`](config/ecp7-b3-development.yaml) — the same utilization formula on straight-through hard messages.
@@ -142,6 +154,10 @@ Do not translate those artifacts in place. New experiment configurations and all
 - [`config/ecp7-b28-development.yaml`](config/ecp7-b28-development.yaml) — Batch 25 with one zero-initialized shared residual sender interaction.
 - [`config/ecp7-b29-development.yaml`](config/ecp7-b29-development.yaml) — Batch 28 with residual parameters frozen through step 30,000.
 - [`config/ecp7-positive-control-development.yaml`](config/ecp7-positive-control-development.yaml) — ECP-6 positive control on the ECP-7 split.
+- [`config/ecp8-positive-control-development.yaml`](config/ecp8-positive-control-development.yaml) — factorized 14-bit positive control on the fresh ECP-8 split.
+- [`config/ecp8-control-development.yaml`](config/ecp8-control-development.yaml) — paired 14-bit weak-structure control.
+- [`config/ecp8-development.yaml`](config/ecp8-development.yaml) — four-global-position 16-bit intervention.
+- [`evidence/ecp8-development/batch1-manifest.json`](evidence/ecp8-development/batch1-manifest.json) — compact sealed Batch 1 evidence and artifact hashes.
 - [`config/ecp0.yaml`](config/ecp0.yaml) — machine-readable configuration from step 1.
 - [`config/ecp1.yaml`](config/ecp1.yaml) — frozen configuration of the population trial.
 - [`config/ecp3.yaml`](config/ecp3.yaml) — frozen configuration of the first successful base model.

config/ecp8-base-development.yaml

@@ -0,0 +1,64 @@
+extends: ecp6-development.yaml
+
+experiment:
+  id: ecp8
+  title: ECP-8 shared sealed development base
+  status: development_test_sealed
+  frozen_at_utc: null
+  parent_experiment: ecp7
+  arm: shared_sealed_base
+  objective: >-
+    Test whether two surplus unallocated channel bits let a weak-structure
+    population learn an injective compositional protocol without restoring a
+    factor-to-slot binding.
+
+reproducibility:
+  dataset_seed: 18072058
+
+dataset:
+  split_strategy: dual_explicit_fresh_color_shape_matchings
+  train_meanings: 14336
+  validation_meanings: 1024
+  compositional_test_meanings: 1024
+  holdout_factors: [color, shape]
+  held_out_factor_pairs: 16
+  validation_held_out_factor_pairs: 16
+  variants_per_held_out_pair: 64
+  excluded_factor_pairs: []
+  explicit_validation_factor_pairs:
+    - {color: 0, shape: 12}
+    - {color: 1, shape: 13}
+    - {color: 2, shape: 14}
+    - {color: 3, shape: 15}
+    - {color: 4, shape: 0}
+    - {color: 5, shape: 1}
+    - {color: 6, shape: 2}
+    - {color: 7, shape: 3}
+    - {color: 8, shape: 4}
+    - {color: 9, shape: 5}
+    - {color: 10, shape: 6}
+    - {color: 11, shape: 7}
+    - {color: 12, shape: 8}
+    - {color: 13, shape: 9}
+    - {color: 14, shape: 10}
+    - {color: 15, shape: 11}
+  explicit_test_factor_pairs:
+    - {color: 0, shape: 13}
+    - {color: 1, shape: 14}
+    - {color: 2, shape: 15}
+    - {color: 3, shape: 0}
+    - {color: 4, shape: 1}
+    - {color: 5, shape: 2}
+    - {color: 6, shape: 3}
+    - {color: 7, shape: 4}
+    - {color: 8, shape: 5}
+    - {color: 9, shape: 6}
+    - {color: 10, shape: 7}
+    - {color: 11, shape: 8}
+    - {color: 12, shape: 9}
+    - {color: 13, shape: 10}
+    - {color: 14, shape: 11}
+    - {color: 15, shape: 12}
+
+outcome_thresholds:
+  paired_control_required: true

config/ecp8-control-development.yaml

@@ -0,0 +1,60 @@
+extends: ecp7-b25-development.yaml
+
+experiment:
+  id: ecp8
+  title: ECP-8 14-bit weak-structure paired control
+  status: development_test_sealed
+  frozen_at_utc: null
+  parent_experiment: ecp7
+  arm: local_capacity_14_bit_control
+  objective: >-
+    Measure the strongest ECP-7 weak-structure system on the fresh ECP-8 split
+    before changing only its channel capacity allocation.
+
+reproducibility:
+  dataset_seed: 18072058
+
+dataset:
+  split_strategy: dual_explicit_fresh_color_shape_matchings
+  train_meanings: 14336
+  validation_meanings: 1024
+  compositional_test_meanings: 1024
+  holdout_factors: [color, shape]
+  held_out_factor_pairs: 16
+  validation_held_out_factor_pairs: 16
+  variants_per_held_out_pair: 64
+  excluded_factor_pairs: []
+  explicit_validation_factor_pairs:
+    - {color: 0, shape: 12}
+    - {color: 1, shape: 13}
+    - {color: 2, shape: 14}
+    - {color: 3, shape: 15}
+    - {color: 4, shape: 0}
+    - {color: 5, shape: 1}
+    - {color: 6, shape: 2}
+    - {color: 7, shape: 3}
+    - {color: 8, shape: 4}
+    - {color: 9, shape: 5}
+    - {color: 10, shape: 6}
+    - {color: 11, shape: 7}
+    - {color: 12, shape: 8}
+    - {color: 13, shape: 9}
+    - {color: 14, shape: 10}
+    - {color: 15, shape: 11}
+  explicit_test_factor_pairs:
+    - {color: 0, shape: 13}
+    - {color: 1, shape: 14}
+    - {color: 2, shape: 15}
+    - {color: 3, shape: 0}
+    - {color: 4, shape: 1}
+    - {color: 5, shape: 2}
+    - {color: 6, shape: 3}
+    - {color: 7, shape: 4}
+    - {color: 8, shape: 5}
+    - {color: 9, shape: 6}
+    - {color: 10, shape: 7}
+    - {color: 11, shape: 8}
+    - {color: 12, shape: 9}
+    - {color: 13, shape: 10}
+    - {color: 14, shape: 11}
+    - {color: 15, shape: 12}

config/ecp8-development.yaml

@@ -0,0 +1,21 @@
+extends: ecp8-control-development.yaml
+
+experiment:
+  title: ECP-8 global-capacity intervention development
+  arm: global_capacity_16_bit_intervention
+  objective: >-
+    Test whether four unrestricted 16-symbol positions eliminate hard-code
+    collisions in the unchanged weak-structure architecture and training setup.
+
+channel:
+  type: discrete_fixed_length
+  vocabulary_size: 16
+  symbol_min: 0
+  symbol_max: 15
+  message_length: 4
+  bits_per_symbol: 4
+  bits_per_message: 16
+  factor_alphabet_sizes: null
+  bits_per_symbol_by_factor: null
+  slot_alphabet_sizes: null
+  bits_per_symbol_by_slot: null

config/ecp8-positive-control-development.yaml

@@ -0,0 +1,8 @@
+extends: ecp8-base-development.yaml
+
+experiment:
+  title: ECP-8 factorized positive-control development
+  arm: ecp6_factorized_positive_control
+  objective: >-
+    Verify that the frozen ECP-6 architecture solves the fresh ECP-8
+    validation matching at the information-theoretic 14-bit lower bound.

docs/AI_AGENT_START.md

@@ -41,7 +41,7 @@ python3.12 -m venv .venv
 .venv/bin/ecp6 --config config/ecp6.yaml validate
 ```
 
-Expected baseline: 133 passing tests and split sizes `14336/1024/1024`.
+Expected baseline: 138 passing tests and split sizes `14336/1024/1024`.
 
 ## 4. Reproduce ECP-6
 
@@ -69,7 +69,7 @@ Then verify hashes, channel constraints and topographic null models:
 
 This reruns a known experiment. It must not be presented as a new independent confirmatory test.
 
-## 5. Continue ECP-7 scientifically
+## 5. Preserve the completed ECP-7 series
 
 Do not modify ECP-0 through ECP-6 or the completed ECP-7 Batch 1 through Batch 29
 configs. ECP7-B1-I collapsed to 130–139 hard messages. ECP7-B2-I improved its
@@ -192,12 +192,26 @@ ECP-7 development is closed after 29 sealed variants. Do not continue a
 residual schedule, collision coefficient, replay route or horizon search, and
 do not open the unused ECP-7 confirmatory split.
 
-Start ECP-8 with a separate preregistration that chooses exactly one new
-question. Plausible directions include learned compositional structure without
-fixed slots, variable-length coding efficiency, interactive negotiation, or
-transfer across altered synthetic worlds. These are different hypotheses:
-select one, define its channel and baselines, generate a fresh deterministic
-split, and register the development and confirmatory gates before training.
+ECP-8 Batch 1 is a separate preregistration that selects the learned-structure
+direction through one narrow capacity question. Read
+`docs/research-design-ecp8.md` and
+`docs/development-log-ecp8.md`. Three seed-11 arms were registered before
+training: a factorized 14-bit positive control, the strongest weak-structure
+14-bit paired control, and the identical weak system with four unrestricted
+16-symbol positions. Do not change their split, seed, architecture, training,
+channel or gates, and never add `--unseal-test` during development.
+
+After all three sealed runs, record the result even if it is negative. A passed
+development gate only permits a later freeze decision; it does not itself
+authorize confirmatory test access.
+
+Batch 1 is complete and negative. The 16-bit arm reached 98.76% mean train,
+74.46% mean validation, 73.34% worst-link validation and 81.05% translator
+validation with 15,095–15,135 unique messages. It failed injectivity and both
+80% validation gates; the confirmatory split remains sealed. Do not search
+larger channels. At most one Batch 2 structure-induction intervention is
+justified at the unchanged 16-bit capacity. Preregister it before training and
+do not combine it with a slot binding or another objective change.
 
 ## 7. Definition of done for any contribution
 

docs/development-log-ecp8.md

@@ -0,0 +1,99 @@
+# ECP-8 Sealed Development Log
+
+The ECP-8 confirmatory split is sealed. This log records every development arm,
+including negative results, before any possible confirmatory access.
+
+## Batch 1 preregistration — Global channel capacity
+
+Preregistered: July 18, 2026 at 19:58:58 UTC<br>
+Seed: `11`<br>
+Test unsealed: **no**<br>
+Split SHA-256: `241acd2ee42ff658ddf0c8c897d4e4df0f52a18098b3ef68c009c74c2134592a`
+
+| Arm | Configuration | Raw SHA-256 | Effective SHA-256 |
+|---|---|---|---|
+| ECP8-B1-P | `config/ecp8-positive-control-development.yaml` | `e996bf66c30b3e699b27046e9824c13c7ecf3f2cb9831c4b9f0528d17b7b5f05` | `3a8e55ee262418f1005405afec3eeee7ebaaad164cb83274bd95c95e2c0939c3` |
+| ECP8-B1-C | `config/ecp8-control-development.yaml` | `9e3c865fcec9698bb95fcfc3ad429a354e26ffd39f2533dd7dc836dfdd3ab34a` | `fcaf5e8f39df2349cb3dedefcc233e5a08d813763a5b644d6a2ed2c050748f06` |
+| ECP8-B1-I | `config/ecp8-development.yaml` | `f6219110ebfe8f26248d9dbd646d80e1033f63d618a2524f7b96668ebd7fdaea` | `0572857aa786e0a41995aedd1cee8c2bb6dbdbc47f46025411bb8c06615b9693` |
+
+ECP8-B1-C reruns the strongest ECP-7 weak-structure setup on the fresh ECP-8
+split. ECP8-B1-I changes only the local channel capacities: `[16,16,8,8]`
+becomes `[16,16,16,16]`, increasing fixed payload size from 14 to 16 bits.
+The sender still has one generic joint context and four position heads; neither
+sender nor receiver receives a factor-slot binding.
+
+The full decision gate is fixed in `docs/research-design-ecp8.md`. In addition
+to the existing train, validation and translator thresholds, the intervention
+must be injective over all 15,360 accessible meanings, improve the paired
+control's minimum codebook, and preserve both mean and worst-link validation.
+Only these three seed-11 runs are admitted. The confirmatory test remains
+unauthorized regardless of a single development result.
+
+## Batch 1 results
+
+Positive-control run: `runs/ecp8-batch1-positive-control-development/20260718T200136Z-ecp8-development`<br>
+Paired-control run: `runs/ecp8-batch1-control-development/20260718T200256Z-ecp8-development`<br>
+Intervention run: `runs/ecp8-batch1-intervention-development/20260718T200637Z-ecp8-development`<br>
+Test unsealed: **no**
+
+| Metric | Positive control | 14-bit control | 16-bit intervention |
+|---|---:|---:|---:|
+| Population train, mean | 100% | 79.5476% | 98.7566% |
+| Population train, worst link | 100% | 78.6551% | 98.6886% |
+| Population validation, mean | 100% | 70.2698% | 74.4629% |
+| Population validation, worst link | 100% | 64.8438% | 73.3398% |
+| Universal translator, validation | 100% | 75.1953% | 81.0547% |
+| Exact sender-message agreement | 100% | 93.74% | 77.56% |
+| Unique messages per sender | 15,360 | 12,048–12,255 | 15,095–15,135 |
+| Collision meanings per sender | 0 | 3,105–3,312 | 225–265 |
+| Message entropy | 13.91 bits | 13.45–13.48 bits | 13.87–13.88 bits |
+| Declared payload | 14 bits | 14 bits | 16 bits |
+| Registered development gate | pass | fail | **fail** |
+
+The positive control passes every validity requirement at 100%, proving that
+the fresh split remains fully solvable under the factorized 14-bit protocol.
+The paired weak-structure control selects step 7,400 and stops at the registered
+15,000-step minimum. The 16-bit intervention keeps improving, selects step
+16,200 and stops at step 21,200.
+
+The extra two bits have a large isolated effect. Mean train exactness improves
+by 19.21 percentage points and worst-link train by 20.03 points. Mean
+validation improves by 4.19 points, worst-link validation by 8.50 points and
+translator validation by 5.86 points. The minimum sender codebook gains 3,047
+occupied messages and the final all-link hard-error replay pool falls from
+1,953 to 119.
+
+Capacity is nevertheless insufficient. The intervention retains 225–265 hard
+collisions per sender, so none is injective. Mean and worst-link validation are
+74.46% and 73.34%, below their registered 80% thresholds. The intervention
+passes both train thresholds, the translator threshold and both paired
+non-regression checks, but fails injectivity and both validation thresholds.
+
+The error pattern also distinguishes memorization from learned compositional
+structure. The 16-bit arm reaches near-perfect training and perfect texture
+validation, while held-out color and shape remain at 94.66% and 79.77% factor
+accuracy. Sender agreement falls from 93.74% to 77.56%, and sender topographic
+correlations fall from 0.761–0.771 to 0.693–0.700. The surplus code space lets
+independent senders separate most training meanings, but it does not induce a
+shared compositional organization for the withheld relation.
+
+All three post-hoc analyses verify 65 artifact hashes, 16,384 validation-only
+episodes, absent confirmatory-test metrics and valid hard messages. The global
+channel audit checks every intervention message for four symbols in `0..15`
+and exactly 16 declared bits.
+
+## Batch 1 decision
+
+ECP8-B1-I is a valid negative development result. Two surplus unallocated bits
+substantially reduce the hard-code bottleneck and improve every joint accuracy
+metric, but they do not produce an injective or sufficiently compositional
+protocol. The pure-capacity hypothesis is rejected, and the ECP-8 confirmatory
+split remains sealed.
+
+Do not continue with an 18-bit, 20-bit or longer fixed message search. A Batch
+2, if attempted, must keep the 16-bit channel and Batch 1 training fixed while
+preregistering exactly one structure-induction mechanism. The most direct
+candidate is the previously defined context-invariant transition objective,
+because Batch 1 now supplies the code capacity that was missing when that
+objective collapsed in ECP7-B8. It must be tested without a slot binding or
+additional simultaneous change.