Skip to content

Commit 9116cd0

Browse files
committed
Record sealed ECP-7 30,000-step batch
1 parent b637ce5 commit 9116cd0

9 files changed

Lines changed: 226 additions & 30 deletions

File tree

‎README.md‎

Lines changed: 7 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -26,15 +26,12 @@ The protocol uses no words or alphabet, but four meaning-free local symbols. For
2626
See [`docs/results-ecp6.md`](docs/results-ecp6.md) for the conclusion, [`docs/protocol-specification-ecp6.md`](docs/protocol-specification-ecp6.md) for the wire format and [`evidence/ecp6/report.md`](evidence/ecp6/report.md) for the compact confirmatory evidence.
2727

2828
ECP-7 development now tests how much of that result depends on the explicit
29-
one-factor-per-slot architecture. Its first fourteen sealed batches are valid
30-
development results but none passes the full gate. Batch 10 remains strongest,
31-
extending the position-aware design to 15,000 optimization steps. It reaches
32-
82.08% train exactness, 72.98% validation and 75.15% translator validation while
33-
using 12,358–12,906 hard messages per sender. The translator threshold now passes,
34-
but the registered train, validation and injectivity thresholds do not. Batch
35-
11's utilization-weight decay regressed all principal measures, and Batch 12's
36-
deeper decoder regressed further. Batch 13's deeper sender collapsed almost
37-
completely, and Batch 14's late learning-rate decay also regressed. The ECP-6
29+
one-factor-per-slot architecture. Its first fifteen sealed batches are valid
30+
development results but none passes the full gate. Batch 15 is now strongest:
31+
the position-aware design at 30,000 optimization steps reaches 83.46% train
32+
exactness, 82.59% validation and 83.37% translator validation while using
33+
12,585–13,200 hard messages per sender. Validation and translator thresholds pass
34+
together, but the registered train and injectivity thresholds do not. The ECP-6
3835
positive controls remain perfect and the ECP-7 confirmatory test remains sealed. See
3936
[`docs/development-log-ecp7.md`](docs/development-log-ecp7.md).
4037

@@ -90,6 +87,7 @@ Do not translate those artifacts in place. New experiment configurations and all
9087
- [`config/ecp7-b12-development.yaml`](config/ecp7-b12-development.yaml) — one additional generic shared decoder layer on the Batch 10 base.
9188
- [`config/ecp7-b13-development.yaml`](config/ecp7-b13-development.yaml) — one additional generic shared sender layer on the Batch 10 base.
9289
- [`config/ecp7-b14-development.yaml`](config/ecp7-b14-development.yaml) — late learning-rate decay on the Batch 10 base.
90+
- [`config/ecp7-b15-development.yaml`](config/ecp7-b15-development.yaml) — the Batch 10 design with a 30,000-step population horizon.
9391
- [`config/ecp7-positive-control-development.yaml`](config/ecp7-positive-control-development.yaml) — ECP-6 positive control on the ECP-7 split.
9492
- [`config/ecp0.yaml`](config/ecp0.yaml) — machine-readable configuration from step 1.
9593
- [`config/ecp1.yaml`](config/ecp1.yaml) — frozen configuration of the population trial.

‎config/ecp7-b15-development.yaml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
extends: ecp7-b10-development.yaml
2+
3+
experiment:
4+
title: ECP-7 position-aware decoder with a 30,000-step horizon
5+
arm: position_aware_decoder_30000_step_optimization
6+
objective: >-
7+
Test whether continued constant-rate optimization beyond the strongest
8+
15,000-step checkpoint can cross the remaining development gates without
9+
changing the architecture, losses, or temperature schedule.
10+
11+
training:
12+
max_steps: 30000
13+
minimum_steps: 15000
14+
patience_steps: 5000

‎docs/AI_AGENT_START.md‎

Lines changed: 21 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ python3.12 -m venv .venv
4141
.venv/bin/ecp6 --config config/ecp6.yaml validate
4242
```
4343

44-
Expected baseline: 88 passing tests and split sizes `14336/1024/1024`.
44+
Expected baseline: 89 passing tests and split sizes `14336/1024/1024`.
4545

4646
## 4. Reproduce ECP-6
4747

@@ -71,7 +71,7 @@ This reruns a known experiment. It must not be presented as a new independent co
7171

7272
## 5. Continue ECP-7 scientifically
7373

74-
Do not modify ECP-0 through ECP-6 or the completed ECP-7 Batch 1 through Batch 14
74+
Do not modify ECP-0 through ECP-6 or the completed ECP-7 Batch 1 through Batch 15
7575
configs. ECP7-B1-I collapsed to 130–139 hard messages. ECP7-B2-I improved its
7676
soft objective but collapsed further to 85–104 hard messages. ECP7-B3-I applied
7777
that loss to straight-through hard messages and improved to 585–972 messages,
@@ -98,15 +98,17 @@ to 79.09% train, 72.22% validation and 72.41% translator validation. B10 remains
9898
the strongest base. ECP7-B12-I added one shared decoder layer and regressed
9999
further to 71.53% train, 47.98% validation and 63.04% translator validation.
100100
ECP7-B13-I added the symmetric sender layer and collapsed to 1.32% train, 0.32%
101-
validation and 0.61% translator validation. The confirmatory ECP-7 test is still
102-
sealed. ECP7-B14-I decayed late learning rate and regressed to 79.13% train,
103-
65.49% validation and 71.02% translator validation. B10 remains the strongest
104-
base.
101+
validation and 0.61% translator validation. ECP7-B14-I decayed late learning
102+
rate and regressed to 79.13% train, 65.49% validation and 71.02% translator
103+
validation. ECP7-B15-I extended constant-rate training to 30,000 steps and is
104+
now strongest at 83.46% train, 82.59% validation and 83.37% translator
105+
validation. Its validation and translator gates pass, but train and injectivity
106+
do not. The confirmatory ECP-7 test is still sealed.
105107

106108
Continue with this sequence:
107109

108110
1. Read `docs/research-design-ecp7.md` and `docs/development-log-ecp7.md`.
109-
2. Define exactly one ECP7-B15 intervention and its failure criterion.
111+
2. Define exactly one ECP7-B16 intervention and its failure criterion.
110112
3. Register the variant and immutable configuration hashes before training.
111113
4. Add tests for every new invariant.
112114
5. Use only `smoke` and `develop`; keep the ECP-7 test split sealed.
@@ -119,25 +121,23 @@ Continue with this sequence:
119121

120122
## 6. Recommended next experiment
121123

122-
Batch 10 is the strongest weak-structure result. Batch 11 ruled out reduced late
123-
utilization pressure, while Batches 12 and 13 ruled out extra generic decoder
124-
and sender depth. Batch 14 showed that reducing late learning rate is also
125-
harmful, while B10 selected its final 15,000-step checkpoint. The most
126-
informative next step is one final duration-only extension. A clean ECP7-B15
127-
progression is:
124+
Batch 15 is the strongest weak-structure result. It passes validation and
125+
translator gates, with residual known-meaning error concentrated in size after
126+
the high-entropy code is established. Earlier Batch 6 minimax pressure started
127+
from step zero on a much weaker protocol. The most informative next step is one
128+
late worst-factor schedule. A clean ECP7-B16 progression is:
128129

129130
- keep the same world and bit budget;
130-
- return to the complete B10 training and loss schedule;
131-
- keep constant learning rate `0.001` and the temperature endpoint `0.8`;
132-
- extend only maximum population steps to 30,000, minimum selection to 15,000
133-
and patience to 5,000;
131+
- keep the complete B15 architecture, horizon and base losses;
132+
- keep normalized factor-minimax weight `0` through step 15,000;
133+
- warm only that weight linearly to `1.0` over steps 15,000–20,000, then hold;
134134
- measure injectivity, validation composition and new-reader induction;
135135
- rerun the ECP-6 architecture as the frozen positive control.
136136

137-
Do not change architecture, loss weights, optimizer, data, temperature,
138-
validation cadence or development thresholds, and do not combine the longer
139-
horizon with variable-length messages or negotiation. That would make its
140-
effect uninterpretable.
137+
Do not change architecture, utilization, optimizer, data, temperature, duration,
138+
validation cadence or development thresholds, and do not combine the late
139+
minimax schedule with variable-length messages or negotiation. That would make
140+
its effect uninterpretable.
141141

142142
## 7. Definition of done for any contribution
143143

‎docs/development-log-ecp7.md‎

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -992,3 +992,79 @@ updates is harmful. A future ECP7-B15 may therefore keep the exact B10
992992
architecture, losses, constant learning rate and temperature endpoint, then
993993
extend only the maximum population horizon to 30,000 steps with minimum
994994
selection step 15,000 and patience 5,000. No other setting may change.
995+
996+
## Batch 15 preregistration
997+
998+
Date: July 18, 2026<br>
999+
Seed: `11`<br>
1000+
Maximum population steps: `30,000`<br>
1001+
Minimum selection steps: `15,000`<br>
1002+
Early-stopping patience: `5,000` steps<br>
1003+
Learning rate: constant `0.001`<br>
1004+
Temperature schedule steps: `5,000`, then hold at `0.8`<br>
1005+
Split-SHA-256: `4947058c75ab07cb43a87eb82776b12cb2a7e2eeba7114de110d3b852cbc64cd`<br>
1006+
Test unsealed: **no**
1007+
1008+
ECP7-B15-I keeps every Batch 10 architecture, loss, optimizer, data and
1009+
translator setting. It extends only population optimization and its selection
1010+
window. The unchanged positive control is rerun, and no alternative horizon or
1011+
accompanying intervention is admitted into this batch.
1012+
1013+
## Batch 15 results
1014+
1015+
Positive-control run: `runs/ecp7-batch15-control-development/20260718T134437Z-ecp7-development`<br>
1016+
Intervention run: `runs/ecp7-batch15-intervention-development/20260718T134437Z-ecp7-development`<br>
1017+
Test unsealed: **no**
1018+
1019+
| Metric | Positive control | ECP7-B15-I |
1020+
|---|---:|---:|
1021+
| Population train, mean | 100% | 83.4612% |
1022+
| Population train, worst link | 100% | 81.9336% |
1023+
| Population validation, mean | 100% | **82.5867%** |
1024+
| Population validation, worst link | 100% | 79.7852% |
1025+
| Universal translator, validation | 100% | **83.3740%** |
1026+
| Exact sender-message agreement | 100% | 92.33% |
1027+
| Unique messages per sender | 15,360 | 12,585–13,200 |
1028+
| Collision meanings per sender | 0 | 2,160–2,775 |
1029+
| Message entropy | 13.91 bits | 13.52–13.61 bits |
1030+
| Development gate | pass | **fail** |
1031+
1032+
The positive control passed every gate at 100%. ECP7-B15-I passed the registered
1033+
mean validation and universal-translator thresholds for the first time in one
1034+
batch. It still failed known-meaning train exactness and sender injectivity, so
1035+
the joint gate fails and confirmatory access remains unauthorized.
1036+
1037+
Relative to B10, the longer constant-rate horizon raised validation from
1038+
72.9797% to 82.5867% and translator validation from 75.1465% to 83.3740%.
1039+
Population train exactness increased only from 82.0836% to 83.4612%, indicating
1040+
that residual hard-message collisions, not held-out composition, are now the
1041+
principal gate failure.
1042+
1043+
Population validation factor accuracies were
1044+
`[98.86%, 99.75%, 98.44%, 85.05%]`; universal-translator factor accuracies were
1045+
`[100%, 100%, 98.44%, 84.94%]`. Color and shape became essentially perfect.
1046+
Size is now the dominant residual factor, with the fixed texture error also
1047+
preventing exact reconstruction.
1048+
1049+
The selected checkpoint was step 28,800 with task loss `0.0675` and utilization
1050+
loss `-0.7399`. Validation first crossed 80% between steps 24,000 and 26,000 and
1051+
remained above it through the final step. Every sender used at least 12,585
1052+
messages, and no collision bucket contained more than three meanings.
1053+
1054+
Both sealed analyses report 65 matching artifact hashes, 16,384 validation-only
1055+
episodes, no confirmatory-test keys, valid local alphabets, and exactly 14
1056+
declared bits for every logged message.
1057+
1058+
## Batch 15 decision
1059+
1060+
ECP7-B15-I is a negative result under the joint gate but becomes the new
1061+
strongest weak-structure base. It passes composition and new-reader induction;
1062+
only known-meaning exactness and injectivity remain below their registered
1063+
thresholds. The confirmatory split remains sealed.
1064+
1065+
The residual error is concentrated in size after the shared code is already
1066+
established. A future ECP7-B16 may keep the complete B15 design and introduce
1067+
only late normalized worst-factor emphasis: factor-minimax weight `0` through
1068+
step 15,000, linearly warm to `1.0` over 5,000 steps, then hold. This revisits the
1069+
Batch 6 objective only after the high-entropy B15 protocol exists and must not
1070+
change architecture, utilization, duration, data, translator or thresholds.

‎docs/research-design-ecp7.md‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -626,3 +626,40 @@ All existing validity and development gates remain unchanged. Failure is
626626
recorded without confirmatory test access. No alternative final rate, decay
627627
shape, decay boundary, architecture, loss, duration, optimizer or seed may be
628628
tried inside Batch 14.
629+
630+
## Batch 15 preregistration — 30,000-step optimization horizon
631+
632+
Status: **preregistered for sealed development**<br>
633+
Preregistered: July 18, 2026<br>
634+
Configuration: `config/ecp7-b15-development.yaml`<br>
635+
Raw configuration SHA-256: `2b4ed0979cb1f6a6aac0409e2624bf982fd020306492706f4aa0e6b3a8c9d15d`<br>
636+
Effective configuration SHA-256: `d60b8a9aebe75514560b35790a3de9674990724fff0a9c52283c91ff429a487b`
637+
638+
Batch 10 selected its final 15,000-step checkpoint. Batch 14 showed that
639+
decaying late learning rate harms validation, while Batches 11–13 rejected loss
640+
removal and added capacity. Batch 15 therefore tests one remaining optimization
641+
hypothesis: the constant-rate B10 trajectory needs more time.
642+
643+
ECP7-B15-I keeps the complete B10 architecture, sender, receiver, translator,
644+
losses, AdamW settings, data, temperature schedule, evaluation cadence and
645+
thresholds. It changes only the population optimization horizon and associated
646+
selection window:
647+
648+
- maximum population steps increase from 15,000 to 30,000;
649+
- minimum selection steps increase from 5,000 to 15,000;
650+
- early-stopping patience increases from 3,000 to 5,000 steps;
651+
- learning rate remains constant at `0.001`;
652+
- temperature still reaches `0.8` at step 5,000 and holds thereafter.
653+
654+
Translator training remains unchanged. Smoke runs are mechanical only and may
655+
not authorize tuning.
656+
657+
The batch contains exactly two seed-11 runs:
658+
659+
1. the unchanged ECP-6 positive control;
660+
2. ECP7-B15-I with the extended B10 population optimization horizon.
661+
662+
All existing validity and development gates remain unchanged. Failure is
663+
recorded without confirmatory test access. No alternative horizon, patience,
664+
learning rate, architecture, loss, temperature, optimizer or seed may be tried
665+
inside Batch 15.

‎evidence/ecp7-development/README.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,11 @@ exactness, 65.49% validation and 71.02% translator validation. Constant late
7878
learning rate is therefore retained. Exact identities and hashes are in
7979
`batch14-manifest.json`.
8080

81+
Batch 15 extended the constant-rate horizon to 30,000 steps. It reached 83.46%
82+
train exactness, 82.59% validation and 83.37% translator validation. Validation
83+
and translator gates passed together, but train exactness and injectivity still
84+
failed. Exact identities and hashes are in `batch15-manifest.json`.
85+
8186
Reproduce the development arms with:
8287

8388
```bash
Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
{
2+
"batch": "ECP7-B15",
3+
"confirmatory_test_unsealed": false,
4+
"control": {
5+
"artifact_hashes": {
6+
"effective_config.yaml": "4bfb976293f117d3ae35f790e4fd6371628be95d91febb334bc275dc5d6b66d4",
7+
"posthoc-analysis.json": "751c592fb1921d6f6fb5d68bd7b7b351523766db3f4aa6be7f1412c3130937a4",
8+
"seed-11/metrics.json": "708b3c84a8c4a6b086b34a55f6d388c637fbf65c076555c7a9912526f5684f47",
9+
"summary.json": "8cb14faab55e787d62dcaabd9ac2215fa100b21dc58d4bfc6b91505348f4c5c0"
10+
},
11+
"development_gate_passed": true,
12+
"population_train_mean_exact_match": 1.0,
13+
"population_validation_mean_exact_match": 1.0,
14+
"run_id": "20260718T134437Z-ecp7-development",
15+
"universal_translator_validation_exact_match": 1.0,
16+
"unique_messages_per_sender": [15360, 15360, 15360, 15360]
17+
},
18+
"intervention": {
19+
"artifact_hashes": {
20+
"effective_config.yaml": "218926e9313ba3dfb7ae3fc52fd3fa8a2f954627253f6d00fbf76109d01aeca5",
21+
"posthoc-analysis.json": "b471cb9c8e69fa5bfaf5bf20eff060924911a31969b8ae0a321e6c126b9a4c6b",
22+
"seed-11/metrics.json": "baeb1840fab41d08655ec4cff90e77d98b04f45ea33cb2140589c79e7f1dd841",
23+
"summary.json": "4565ca20b499167f99ed2463185d097c4e6fbe563611682141630300036a1ff0"
24+
},
25+
"best_code_utilization_loss": -0.7398611903190613,
26+
"best_step": 28800,
27+
"collision_meanings_per_sender": [2520, 2490, 2775, 2160],
28+
"development_gate_passed": false,
29+
"exact_sender_message_agreement": 0.9232530381944444,
30+
"message_entropy_bits_per_sender": [13.56107291977656, 13.56497916977656, 13.516074677555256, 13.60794791977656],
31+
"population_train_mean_exact_match": 0.8346121651785714,
32+
"population_train_worst_link_exact_match": 0.8193359375,
33+
"population_validation_factor_accuracy": [0.9886474609375, 0.99749755859375, 0.984375, 0.8504638671875],
34+
"population_validation_mean_exact_match": 0.82586669921875,
35+
"population_validation_worst_link_exact_match": 0.7978515625,
36+
"run_id": "20260718T134437Z-ecp7-development",
37+
"universal_translator_validation_exact_match": 0.833740234375,
38+
"unique_messages_per_sender": [12840, 12870, 12585, 13200]
39+
},
40+
"seed": 11,
41+
"split_artifact_sha256": "676195997ee0c77d5c9565bae045dbb938b432a2e4e74b666233b0a41f924718",
42+
"split_sha256": "4947058c75ab07cb43a87eb82776b12cb2a7e2eeba7114de110d3b852cbc64cd"
43+
}

‎tests/conftest.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -113,3 +113,8 @@ def ecp7_b13_config():
113113
@pytest.fixture(scope="session")
114114
def ecp7_b14_config():
115115
return load_config(PROJECT_ROOT / "config" / "ecp7-b14-development.yaml")
116+
117+
118+
@pytest.fixture(scope="session")
119+
def ecp7_b15_config():
120+
return load_config(PROJECT_ROOT / "config" / "ecp7-b15-development.yaml")

‎tests/test_config.py‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -511,3 +511,21 @@ def test_learning_rate_decay_must_be_bounded(ecp7_b14_config):
511511
] = 0.001
512512
with pytest.raises(ConfigError, match="below its initial value"):
513513
validate_config(invalid_final)
514+
515+
516+
def test_ecp7_b15_changes_only_the_population_optimization_horizon(
517+
ecp7_b10_config, ecp7_b15_config
518+
):
519+
assert ecp7_b15_config["world"] == ecp7_b10_config["world"]
520+
assert ecp7_b15_config["dataset"] == ecp7_b10_config["dataset"]
521+
assert ecp7_b15_config["channel"] == ecp7_b10_config["channel"]
522+
assert ecp7_b15_config["agents"] == ecp7_b10_config["agents"]
523+
b10_training = deepcopy(ecp7_b10_config["training"])
524+
b15_training = deepcopy(ecp7_b15_config["training"])
525+
assert b10_training.pop("max_steps") == 15000
526+
assert b15_training.pop("max_steps") == 30000
527+
assert b10_training.pop("minimum_steps") == 5000
528+
assert b15_training.pop("minimum_steps") == 15000
529+
assert b10_training.pop("patience_steps") == 3000
530+
assert b15_training.pop("patience_steps") == 5000
531+
assert b15_training == b10_training

0 commit comments

Comments
 (0)