Skip to content

Commit 9c347df

Browse files
committed
Add ECP-7 Batch 25 horizon study
1 parent 9eb8283 commit 9c347df

9 files changed

Lines changed: 269 additions & 17 deletions

File tree

‎README.md‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ The protocol uses no words or alphabet, but four meaning-free local symbols. For
2626
See [`docs/results-ecp6.md`](docs/results-ecp6.md) for the conclusion, [`docs/protocol-specification-ecp6.md`](docs/protocol-specification-ecp6.md) for the wire format and [`evidence/ecp6/report.md`](evidence/ecp6/report.md) for the compact confirmatory evidence.
2727

2828
ECP-7 development now tests how much of that result depends on the explicit
29-
one-factor-per-slot architecture. Its first twenty-four sealed batches are valid
29+
one-factor-per-slot architecture. Its first twenty-five sealed batches are valid
3030
development results but none passes the full gate. Batch 15 established the
3131
strong position-aware base: at 30,000 optimization steps it reaches 83.46% train
3232
exactness, 82.59% validation and 83.37% translator validation while using
@@ -51,7 +51,11 @@ sender-only warmup. Validation remained 83.50%; total fragmented errors fell,
5151
but the shared-error pool immediately regrew and injectivity still failed.
5252
Batch 24 made the second phase receiver-only. It established new-best means of
5353
84.69% train, 84.48% validation and 84.55% translator validation, but the
54-
worst-link and injectivity gates still failed. See
54+
worst-link and injectivity gates still failed.
55+
Batch 25 extended that catch-up to 45,000 steps and improved again to 85.26%
56+
train, 85.24% validation and 85.13% translator validation. Receiver performance
57+
now nearly reaches the mathematical ceiling of the non-injective sender
58+
codebooks, so another horizon-only extension is not justified. See
5559
[`docs/development-log-ecp7.md`](docs/development-log-ecp7.md).
5660

5761
## Build on this work
@@ -116,6 +120,7 @@ Do not translate those artifacts in place. New experiment configurations and all
116120
- [`config/ecp7-b22-development.yaml`](config/ecp7-b22-development.yaml) — Batch 21 replay with receiver-parameter gradients disabled only on the replay branch.
117121
- [`config/ecp7-b23-development.yaml`](config/ecp7-b23-development.yaml) — Batch 22 sender-only replay through step 20,000 followed by restored joint replay gradients.
118122
- [`config/ecp7-b24-development.yaml`](config/ecp7-b24-development.yaml) — Batch 23 routing with sender messages detached for receiver-only replay after step 20,000.
123+
- [`config/ecp7-b25-development.yaml`](config/ecp7-b25-development.yaml) — Batch 24 receiver-only catch-up with an isolated 45,000-step maximum horizon.
119124
- [`config/ecp7-positive-control-development.yaml`](config/ecp7-positive-control-development.yaml) — ECP-6 positive control on the ECP-7 split.
120125
- [`config/ecp0.yaml`](config/ecp0.yaml) — machine-readable configuration from step 1.
121126
- [`config/ecp1.yaml`](config/ecp1.yaml) — frozen configuration of the population trial.

‎config/ecp7-b25-development.yaml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
extends: ecp7-b24-development.yaml
2+
3+
experiment:
4+
title: ECP-7 extended receiver-only catch-up horizon
5+
arm: extended_receiver_only_catchup_horizon
6+
objective: >-
7+
Test whether the late receiver-only catch-up phase continues improving
8+
reconstruction or reaches a non-injective plateau when given enough time
9+
for the registered early-stopping patience to operate.
10+
11+
training:
12+
max_steps: 45000

‎docs/AI_AGENT_START.md‎

Lines changed: 22 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ python3.12 -m venv .venv
4141
.venv/bin/ecp6 --config config/ecp6.yaml validate
4242
```
4343

44-
Expected baseline: 121 passing tests and split sizes `14336/1024/1024`.
44+
Expected baseline: 122 passing tests and split sizes `14336/1024/1024`.
4545

4646
## 4. Reproduce ECP-6
4747

@@ -71,7 +71,7 @@ This reruns a known experiment. It must not be presented as a new independent co
7171

7272
## 5. Continue ECP-7 scientifically
7373

74-
Do not modify ECP-0 through ECP-6 or the completed ECP-7 Batch 1 through Batch 24
74+
Do not modify ECP-0 through ECP-6 or the completed ECP-7 Batch 1 through Batch 25
7575
configs. ECP7-B1-I collapsed to 130–139 hard messages. ECP7-B2-I improved its
7676
soft objective but collapsed further to 85–104 hard messages. ECP7-B3-I applied
7777
that loss to straight-through hard messages and improved to 585–972 messages,
@@ -144,11 +144,16 @@ ECP7-B24-I made the post-warmup replay phase receiver-only. It set new ECP-7
144144
bests of 84.69% mean train, 84.48% mean validation and 84.55% translator
145145
validation, but worst-link validation remained 82.23%, sender injectivity
146146
failed and the confirmatory ECP-7 test remains sealed.
147+
ECP7-B25-I extended only that catch-up horizon to 45,000 steps. It improved to
148+
85.26% mean train, 85.24% mean validation, 82.71% worst-link validation and
149+
85.13% translator validation. The worst train link exactly matches its 82.81%
150+
unique-code ceiling, so sender injectivity remains the structural blocker and
151+
the confirmatory ECP-7 test remains sealed.
147152

148153
Continue with this sequence:
149154

150155
1. Read `docs/research-design-ecp7.md` and `docs/development-log-ecp7.md`.
151-
2. Define exactly one ECP7-B25 intervention and its failure criterion.
156+
2. Define exactly one ECP7-B26 intervention and its failure criterion.
152157
3. Register the variant and immutable configuration hashes before training.
153158
4. Add tests for every new invariant.
154159
5. Use only `smoke` and `develop`; keep the ECP-7 test split sealed.
@@ -161,26 +166,28 @@ Continue with this sequence:
161166

162167
## 6. Recommended next experiment
163168

164-
Batch 24 confirms that receiver-only replay catch-up improves all mean metrics
165-
but retains cross-link fragmentation and non-injective sender codes. Its best
166-
checkpoint occurs at step 29,400, only 600 steps before the fixed horizon, so
167-
the registered patience could not test a late plateau. A clean ECP7-B25
168-
progression is:
169-
170-
- inherit the complete Batch 24 implementation and configuration;
171-
- change only `training.max_steps` from `30,000` to `45,000`;
172-
- keep the 5,000-step early-stopping patience and minimum selection step
173-
unchanged, so the run may stop before 45,000 if a genuine plateau occurs;
169+
Batch 25 confirms that extra receiver-only catch-up time improves decoder fit
170+
but leaves the sender collision structure almost unchanged. The observed worst
171+
train link exactly reaches its sender's unique-code ceiling, so another horizon
172+
extension cannot solve the gate. A clean ECP7-B26 coordinate-descent progression
173+
is:
174+
175+
- inherit the complete Batch 25 implementation and 45,000-step ceiling;
176+
- keep sender-only replay through step 20,000 and receiver-only replay through
177+
step 30,000 exactly as in Batch 25;
178+
- route the additional replay branch sender-only again for steps 30,001–35,000,
179+
then restore receiver-only replay from step 35,001;
180+
- reuse the registered 5,000-step sender-separation duration;
174181
- keep ordinary base-task updates to both senders and receivers unchanged;
175182
- keep weight `0.25`, the 64-meaning replay batch, 200-step refresh,
176183
seed-derived sampler, predicate and temperature unchanged;
177184
- continue to mine and replay training meanings only;
178185
- measure shared and any-link pool sizes, train exactness, injectivity,
179186
validation and new-reader induction, and rerun the frozen ECP-6 control.
180187

181-
Do not combine the horizon change with another predicate, coefficient, graded
188+
Do not combine the routing cycle with another predicate, coefficient, graded
182189
sampling, collision pressure, a new architecture, factor-specific weights,
183-
optimizer changes, a second horizon change, variable-length messages or
190+
optimizer changes, a horizon change, variable-length messages or
184191
negotiation.
185192

186193
## 7. Definition of done for any contribution

‎docs/development-log-ecp7.md‎

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ this log.
3232
| ECP7-B22-I | Intervention | B21 replay gradients routed to senders but not receiver parameters | Position-aware MLP | valid negative; shared errors fall but link-specific fragmentation rises |
3333
| ECP7-B23-I | Intervention | B22 sender-only replay warmup followed by restored joint replay gradients | Position-aware MLP | valid negative; joint phase recreates shared errors without improving accuracy |
3434
| ECP7-B24-I | Intervention | B23 sender-only warmup followed by receiver-only replay catch-up | Position-aware MLP | valid negative; new-best means but link-specific errors and injectivity fail |
35+
| ECP7-B25-I | Intervention | B24 receiver-only catch-up with a 45,000-step maximum horizon | Position-aware MLP | valid negative; new-best means reach the non-injective code-cap |
3536

3637
## Batch 1 preregistration
3738

@@ -1982,3 +1983,108 @@ A future ECP7-B25 may inherit Batch 24 unchanged and extend only `max_steps` to
19821983
45,000. This isolates whether the receiver-only phase is still improving or
19831984
merely approaching a non-injective plateau. No coefficient, route, predicate,
19841985
architecture, optimizer or data setting may change with that horizon test.
1986+
1987+
## Batch 25 preregistration
1988+
1989+
Preregistered: July 18, 2026 at 17:26:20 UTC<br>
1990+
Seed: `11`<br>
1991+
Maximum population steps: `45,000`<br>
1992+
Minimum selection steps: `15,000`<br>
1993+
Early-stopping patience: `5,000` steps<br>
1994+
Hard-meaning predicate: all `16` current population links have an incorrect factor<br>
1995+
Replay sender-message gradients: **enabled through step 20,000; disabled from step 20,001**<br>
1996+
Replay receiver-parameter gradients: **disabled through step 20,000; enabled from step 20,001**<br>
1997+
Ordinary-task sender and receiver gradients: **enabled throughout**<br>
1998+
Replay weight: `0` through step `15,000`, linear to `0.25` at step `20,000`, then hold<br>
1999+
Replay batch size: `64` meanings, uniform with replacement<br>
2000+
Training-only pool refresh interval: `200` steps<br>
2001+
Replay sampler seed offset: `+1211`<br>
2002+
Raw configuration SHA-256: `3c125aa0e21b29327b332a77f0a367633267804523eb941d5dad9eb622255387`<br>
2003+
Effective configuration SHA-256: `00c9f13c5919f98f5b81c14c8ea2d852a1cd4be55ecbfcaf0d61abed6b9acc91`<br>
2004+
Split-SHA-256: `4947058c75ab07cb43a87eb82776b12cb2a7e2eeba7114de110d3b852cbc64cd`<br>
2005+
Test unsealed: **no**
2006+
2007+
ECP7-B25-I inherits Batch 24 and changes only `training.max_steps` from
2008+
`30,000` to `45,000`. The registered 5,000-step early-stopping patience remains
2009+
active, so 45,000 is a ceiling rather than a required duration. All replay
2010+
routes, timing, weights, data and selection criteria remain unchanged. The
2011+
positive control is rerun, and no alternative horizon or accompanying
2012+
intervention is admitted into this batch.
2013+
2014+
## Batch 25 results
2015+
2016+
Positive-control run: `runs/ecp7-batch25-control-development/20260718T172746Z-ecp7-development`<br>
2017+
Intervention run: `runs/ecp7-batch25-intervention-development/20260718T172746Z-ecp7-development`<br>
2018+
Test unsealed: **no**
2019+
2020+
| Metric | Positive control | ECP7-B25-I |
2021+
|---|---:|---:|
2022+
| Population train, mean | 100% | **85.2631%** |
2023+
| Population train, worst link | 100% | 82.8125% |
2024+
| Population validation, mean | 100% | **85.2417%** |
2025+
| Population validation, worst link | 100% | 82.7148% |
2026+
| Universal translator, validation | 100% | **85.1318%** |
2027+
| Exact sender-message agreement | 100% | 92.16% |
2028+
| Unique messages per sender | 15,360 | 12,720–13,440 |
2029+
| Collision meanings per sender | 0 | 1,920–2,640 |
2030+
| Message entropy | 13.91 bits | 13.54–13.64 bits |
2031+
| Development gate | pass | **fail** |
2032+
2033+
The positive control passed every gate at 100%. ECP7-B25-I improves every
2034+
primary B24 accuracy and sender agreement, setting new ECP-7 development bests
2035+
for mean and worst-link train, mean and worst-link validation, and translator
2036+
validation. It still fails both train thresholds and sender injectivity. The
2037+
joint gate fails and confirmatory access remains unauthorized.
2038+
2039+
All 151 optimization records through step 30,000 match Batch 24 exactly. The
2040+
extended receiver-only phase raises mean validation from 84.45% at step 30,000
2041+
to the selected 85.24% at step 43,800. Training reaches the 45,000-step ceiling
2042+
with 85.09% validation, so the registered patience still does not terminate the
2043+
run before the cap. Improvement after step 30,000 is real but slow and noisy.
2044+
2045+
The all-link pool contains 1,710 meanings at step 30,000, 1,691 at step 40,000,
2046+
1,677 at selection and 1,659 at the final step. The selected diagnostic finds
2047+
1,677 meanings failed by all links and 2,967 failed by at least one link, with
2048+
11.39 mean failed links per erroneous meaning. Relative to B24, the longer
2049+
phase removes 175 any-link failures but only 13 shared failures.
2050+
2051+
The selected codebooks explain the plateau. Their unique-message counts are
2052+
13,440, 12,960, 12,720 and 13,440. Even a perfect receiver can therefore decode
2053+
at most 87.50%, 84.38%, 82.81% and 87.50% of each sender's 15,360 training
2054+
meanings. The mean code-cap is 85.55%; observed mean train is already 85.26%,
2055+
and the observed worst link exactly equals the lowest 82.81% code-cap. Longer
2056+
receiver-only optimization cannot satisfy the train gates without first
2057+
changing sender collisions.
2058+
2059+
Compared with Batch 24, B25 improves mean train by 0.58 percentage points,
2060+
worst-link train by 0.25 points, mean validation by 0.76 points, worst-link
2061+
validation by 0.49 points, translator validation by 0.59 points and sender
2062+
agreement by 0.66 points. Receiver induction has caught up to the existing
2063+
codes, but the remaining performance gap is structurally sender-side.
2064+
2065+
Selected sender codebooks contain 13,440, 12,960, 12,720 and 13,440 unique
2066+
messages, with 1,920, 2,400, 2,640 and 1,920 collision meanings. Population
2067+
validation factor accuracies were `[99.99%, 100%, 98.44%, 86.82%]`;
2068+
universal-translator factor accuracies were
2069+
`[100%, 100%, 98.44%, 86.69%]`. The stable size collisions and sender-specific
2070+
texture caps remain the limiting structure.
2071+
2072+
Both sealed analyses report 65 matching artifact hashes, 16,384 validation-only
2073+
episodes, no confirmatory-test keys, valid local alphabets, and exactly 14
2074+
declared bits for every logged message.
2075+
2076+
## Batch 25 decision
2077+
2078+
ECP7-B25-I is a valid negative result and the strongest ECP-7 development
2079+
variant so far. The extended catch-up phase improves receiver fit but reaches
2080+
the mathematical ceiling imposed by non-injective sender codebooks. The
2081+
confirmatory split remains sealed, and another horizon-only extension is not
2082+
justified.
2083+
2084+
A future ECP7-B26 may test one deterministic coordinate-descent cycle while
2085+
keeping Batch 25's 45,000-step ceiling: sender-only replay through step 20,000,
2086+
receiver-only through 30,000, a second sender-only separation pulse through
2087+
35,000, and receiver-only catch-up afterward. The second sender pulse reuses the
2088+
registered 5,000-step duration and directly tests whether saturated receivers
2089+
provide a better fixed target for breaking remaining sender collisions. No
2090+
other setting may change.

‎docs/research-design-ecp7.md‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1053,3 +1053,41 @@ sealed-test requirement fails. No alternative switch step, gradient route,
10531053
predicate, coefficient, replay batch, refresh, architecture, base loss,
10541054
optimizer, duration, data, translator or seed may be tried inside Batch 24.
10551055
Failure is recorded without confirmatory test access.
1056+
1057+
## Batch 25 preregistration — Extended receiver-only catch-up horizon
1058+
1059+
Status: **preregistered for sealed development**<br>
1060+
Preregistered: July 18, 2026 at 17:26:20 UTC<br>
1061+
Configuration: `config/ecp7-b25-development.yaml`<br>
1062+
Raw configuration SHA-256: `3c125aa0e21b29327b332a77f0a367633267804523eb941d5dad9eb622255387`<br>
1063+
Effective configuration SHA-256: `00c9f13c5919f98f5b81c14c8ea2d852a1cd4be55ecbfcaf0d61abed6b9acc91`
1064+
1065+
Batch 24 established the strongest mean ECP-7 performance, selected its best
1066+
checkpoint at step 29,400, and reached the 30,000-step ceiling before the
1067+
5,000-step patience could assess a late plateau. Batch 25 isolates whether that
1068+
receiver-only catch-up phase is still improving or has already converged to a
1069+
non-injective solution.
1070+
1071+
ECP7-B25-I inherits the complete Batch 24 implementation, configuration,
1072+
gradient routes, hard-meaning predicate, replay mechanism, architecture,
1073+
ordinary task batch, utilization objective, optimizer, data, temperature
1074+
schedule, coefficient, replay batch, sampler, refresh, warmup, selection,
1075+
translator and thresholds. The sole change is:
1076+
1077+
- `training.max_steps` increases from 30,000 to 45,000.
1078+
1079+
The 5,000-step early-stopping patience and 15,000-step minimum remain unchanged.
1080+
The run may therefore finish before step 45,000. The complete optimization
1081+
trajectory through step 30,000 must match Batch 24 exactly.
1082+
1083+
The batch contains exactly two seed-11 runs:
1084+
1085+
1. the unchanged ECP-6 positive control;
1086+
2. ECP7-B25-I with the extended receiver-only catch-up ceiling.
1087+
1088+
All existing validity and development gates remain unchanged. The intervention
1089+
fails if any train, validation, translator, injectivity, channel-integrity or
1090+
sealed-test requirement fails. No alternative horizon, gradient route,
1091+
predicate, coefficient, replay batch, refresh, architecture, base loss,
1092+
optimizer, data, translator or seed may be tried inside Batch 25. Failure is
1093+
recorded without confirmatory test access.

‎evidence/ecp7-development/README.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -136,6 +136,13 @@ validation. The selected error pool contains 1,690 shared failures and 3,142
136136
any-link failures, so worst-link accuracy and injectivity still fail. Exact
137137
identities and hashes are in `batch24-manifest.json`.
138138

139+
Batch 25 extended only the receiver-only catch-up ceiling to 45,000 steps. It
140+
set new-best means of 85.26% train, 85.24% validation and 85.13% translator
141+
validation, but the observed worst train link exactly reached its sender's
142+
82.81% unique-code ceiling. The remaining failure is sender injectivity, not
143+
receiver training time. Exact identities and hashes are in
144+
`batch25-manifest.json`.
145+
139146
Reproduce the development arms with:
140147

141148
```bash

0 commit comments

Comments
 (0)