Skip to content

Commit 9eb8283

Browse files
authored
Add ECP-7 Batch 24 receiver catch-up study (#25)
Document the sealed receiver-only catch-up experiment and its valid negative result.
1 parent 03f28cd commit 9eb8283

12 files changed

Lines changed: 434 additions & 19 deletions

File tree

‎README.md‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ The protocol uses no words or alphabet, but four meaning-free local symbols. For
2626
See [`docs/results-ecp6.md`](docs/results-ecp6.md) for the conclusion, [`docs/protocol-specification-ecp6.md`](docs/protocol-specification-ecp6.md) for the wire format and [`evidence/ecp6/report.md`](evidence/ecp6/report.md) for the compact confirmatory evidence.
2727

2828
ECP-7 development now tests how much of that result depends on the explicit
29-
one-factor-per-slot architecture. Its first twenty-three sealed batches are valid
29+
one-factor-per-slot architecture. Its first twenty-four sealed batches are valid
3030
development results but none passes the full gate. Batch 15 established the
3131
strong position-aware base: at 30,000 optimization steps it reaches 83.46% train
3232
exactness, 82.59% validation and 83.37% translator validation while using
@@ -48,7 +48,10 @@ confirmatory test remains sealed. Batch 22 routed shared-error replay only into
4848
senders; this reduced shared errors but increased cross-link fragmentation and
4949
regressed validation to 83.51%. Batch 23 restored joint replay after the
5050
sender-only warmup. Validation remained 83.50%; total fragmented errors fell,
51-
but the shared-error pool immediately regrew and injectivity still failed. See
51+
but the shared-error pool immediately regrew and injectivity still failed.
52+
Batch 24 made the second phase receiver-only. It established new-best means of
53+
84.69% train, 84.48% validation and 84.55% translator validation, but the
54+
worst-link and injectivity gates still failed. See
5255
[`docs/development-log-ecp7.md`](docs/development-log-ecp7.md).
5356

5457
## Build on this work
@@ -112,6 +115,7 @@ Do not translate those artifacts in place. New experiment configurations and all
112115
- [`config/ecp7-b21-development.yaml`](config/ecp7-b21-development.yaml) — Batch 20 replay restricted to meanings failed by all 16 population links.
113116
- [`config/ecp7-b22-development.yaml`](config/ecp7-b22-development.yaml) — Batch 21 replay with receiver-parameter gradients disabled only on the replay branch.
114117
- [`config/ecp7-b23-development.yaml`](config/ecp7-b23-development.yaml) — Batch 22 sender-only replay through step 20,000 followed by restored joint replay gradients.
118+
- [`config/ecp7-b24-development.yaml`](config/ecp7-b24-development.yaml) — Batch 23 routing with sender messages detached for receiver-only replay after step 20,000.
115119
- [`config/ecp7-positive-control-development.yaml`](config/ecp7-positive-control-development.yaml) — ECP-6 positive control on the ECP-7 split.
116120
- [`config/ecp0.yaml`](config/ecp0.yaml) — machine-readable configuration from step 1.
117121
- [`config/ecp1.yaml`](config/ecp1.yaml) — frozen configuration of the population trial.

‎config/ecp7-b24-development.yaml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
extends: ecp7-b23-development.yaml
2+
3+
experiment:
4+
title: ECP-7 receiver-only hard-replay catch-up
5+
arm: receiver_only_hard_replay_catchup
6+
objective: >-
7+
Test whether a sender-only replay warmup followed by receiver-only replay
8+
can preserve separated sender codes while allowing receivers to catch up
9+
without joint replay coadaptation.
10+
11+
training:
12+
global_hard_meaning_replay:
13+
disable_sender_message_gradients_after_step: 20000

‎docs/AI_AGENT_START.md‎

Lines changed: 20 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ python3.12 -m venv .venv
4141
.venv/bin/ecp6 --config config/ecp6.yaml validate
4242
```
4343

44-
Expected baseline: 117 passing tests and split sizes `14336/1024/1024`.
44+
Expected baseline: 121 passing tests and split sizes `14336/1024/1024`.
4545

4646
## 4. Reproduce ECP-6
4747

@@ -71,7 +71,7 @@ This reruns a known experiment. It must not be presented as a new independent co
7171

7272
## 5. Continue ECP-7 scientifically
7373

74-
Do not modify ECP-0 through ECP-6 or the completed ECP-7 Batch 1 through Batch 23
74+
Do not modify ECP-0 through ECP-6 or the completed ECP-7 Batch 1 through Batch 24
7575
configs. ECP7-B1-I collapsed to 130–139 hard messages. ECP7-B2-I improved its
7676
soft objective but collapsed further to 85–104 hard messages. ECP7-B3-I applied
7777
that loss to straight-through hard messages and improved to 585–972 messages,
@@ -140,11 +140,15 @@ shared-error pool reached 1,414 during sender-only routing, then jumped to 1,984
140140
at the first joint boundary and reached 2,085 at selection. The any-link pool
141141
fell to 2,660, but validation remained 83.50%, train exactness and injectivity
142142
still failed, and the confirmatory ECP-7 test remains sealed.
143+
ECP7-B24-I made the post-warmup replay phase receiver-only. It set new ECP-7
144+
bests of 84.69% mean train, 84.48% mean validation and 84.55% translator
145+
validation, but worst-link validation remained 82.23%, sender injectivity
146+
failed and the confirmatory ECP-7 test remains sealed.
143147

144148
Continue with this sequence:
145149

146150
1. Read `docs/research-design-ecp7.md` and `docs/development-log-ecp7.md`.
147-
2. Define exactly one ECP7-B24 intervention and its failure criterion.
151+
2. Define exactly one ECP7-B25 intervention and its failure criterion.
148152
3. Register the variant and immutable configuration hashes before training.
149153
4. Add tests for every new invariant.
150154
5. Use only `smoke` and `develop`; keep the ECP-7 test split sealed.
@@ -157,26 +161,27 @@ Continue with this sequence:
157161

158162
## 6. Recommended next experiment
159163

160-
Batch 23 confirms that restoring joint replay after sender-only warmup promptly
161-
recreates shared errors. Batch 22 confirms that staying sender-only instead
162-
creates cross-link fragmentation. The clean remaining routing question is
163-
whether receivers can catch up without letting the replay branch move sender
164-
codes again. A clean ECP7-B24 progression is:
164+
Batch 24 confirms that receiver-only replay catch-up improves all mean metrics
165+
but retains cross-link fragmentation and non-injective sender codes. Its best
166+
checkpoint occurs at step 29,400, only 600 steps before the fixed horizon, so
167+
the registered patience could not test a late plateau. A clean ECP7-B25
168+
progression is:
165169

166-
- inherit the complete Batch 23 implementation, Batch 21 predicate and Batch 15
167-
base;
168-
- keep replay sender-only during steps 15,000–20,000, then make the additional
169-
replay branch receiver-only by detaching sender messages after step 20,000;
170+
- inherit the complete Batch 24 implementation and configuration;
171+
- change only `training.max_steps` from `30,000` to `45,000`;
172+
- keep the 5,000-step early-stopping patience and minimum selection step
173+
unchanged, so the run may stop before 45,000 if a genuine plateau occurs;
170174
- keep ordinary base-task updates to both senders and receivers unchanged;
171175
- keep weight `0.25`, the 64-meaning replay batch, 200-step refresh,
172-
seed-derived sampler, predicate, temperature and horizon unchanged;
176+
seed-derived sampler, predicate and temperature unchanged;
173177
- continue to mine and replay training meanings only;
174178
- measure shared and any-link pool sizes, train exactness, injectivity,
175179
validation and new-reader induction, and rerun the frozen ECP-6 control.
176180

177-
Do not combine the routing schedule with another predicate, coefficient, graded
181+
Do not combine the horizon change with another predicate, coefficient, graded
178182
sampling, collision pressure, a new architecture, factor-specific weights,
179-
optimizer changes, longer training, variable-length messages or negotiation.
183+
optimizer changes, a second horizon change, variable-length messages or
184+
negotiation.
180185

181186
## 7. Definition of done for any contribution
182187

‎docs/development-log-ecp7.md‎

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ this log.
3131
| ECP7-B21-I | Intervention | B20 replay restricted to meanings failed by all 16 links | Position-aware MLP | valid negative; new-best validation 83.63%, shared-error pool grows |
3232
| ECP7-B22-I | Intervention | B21 replay gradients routed to senders but not receiver parameters | Position-aware MLP | valid negative; shared errors fall but link-specific fragmentation rises |
3333
| ECP7-B23-I | Intervention | B22 sender-only replay warmup followed by restored joint replay gradients | Position-aware MLP | valid negative; joint phase recreates shared errors without improving accuracy |
34+
| ECP7-B24-I | Intervention | B23 sender-only warmup followed by receiver-only replay catch-up | Position-aware MLP | valid negative; new-best means but link-specific errors and injectivity fail |
3435

3536
## Batch 1 preregistration
3637

@@ -1875,3 +1876,109 @@ phase receiver-only rather than joint. Detaching replay messages after step
18751876
20,000 would let receivers catch up to the sender-only code movement without
18761877
allowing the replay branch to move those codes again. The ordinary task must
18771878
remain joint throughout, and no other setting may change.
1879+
1880+
## Batch 24 preregistration
1881+
1882+
Preregistered: July 18, 2026 at 17:09:59 UTC<br>
1883+
Seed: `11`<br>
1884+
Maximum population steps: `30,000`<br>
1885+
Minimum selection steps: `15,000`<br>
1886+
Early-stopping patience: `5,000` steps<br>
1887+
Hard-meaning predicate: all `16` current population links have an incorrect factor<br>
1888+
Replay sender-message gradients: **enabled through step 20,000; disabled from step 20,001**<br>
1889+
Replay receiver-parameter gradients: **disabled through step 20,000; enabled from step 20,001**<br>
1890+
Ordinary-task sender and receiver gradients: **enabled throughout**<br>
1891+
Replay weight: `0` through step `15,000`, linear to `0.25` at step `20,000`, then hold<br>
1892+
Replay batch size: `64` meanings, uniform with replacement<br>
1893+
Training-only pool refresh interval: `200` steps<br>
1894+
Replay sampler seed offset: `+1211`<br>
1895+
Raw configuration SHA-256: `27e737c1eaa3c191253469fb863171c9875e1e9433c2b00b7a578e15609155dc`<br>
1896+
Effective configuration SHA-256: `a01f184249b32859074909039ba86d495854a3c5f063c19f5519e2739135cb9e`<br>
1897+
Split-SHA-256: `4947058c75ab07cb43a87eb82776b12cb2a7e2eeba7114de110d3b852cbc64cd`<br>
1898+
Test unsealed: **no**
1899+
1900+
ECP7-B24-I inherits Batch 23 and changes only sender-message gradient routing
1901+
for the additional replay branch after step 20,000. Replay is sender-only
1902+
through the registered warmup and receiver-only afterward. Receiver-only replay
1903+
detaches the sampled sender messages before reconstruction, while the ordinary
1904+
task continues to update both senders and receivers. Gradient invariant tests
1905+
cover each isolated replay phase and both ordinary-task routes. The positive
1906+
control is rerun, and no alternative switch point, routing, predicate or
1907+
accompanying intervention is admitted into this batch.
1908+
1909+
## Batch 24 results
1910+
1911+
Positive-control run: `runs/ecp7-batch24-control-development/20260718T171239Z-ecp7-development`<br>
1912+
Intervention run: `runs/ecp7-batch24-intervention-development/20260718T171239Z-ecp7-development`<br>
1913+
Test unsealed: **no**
1914+
1915+
| Metric | Positive control | ECP7-B24-I |
1916+
|---|---:|---:|
1917+
| Population train, mean | 100% | **84.6854%** |
1918+
| Population train, worst link | 100% | 82.5614% |
1919+
| Population validation, mean | 100% | **84.4849%** |
1920+
| Population validation, worst link | 100% | 82.2266% |
1921+
| Universal translator, validation | 100% | **84.5459%** |
1922+
| Exact sender-message agreement | 100% | 91.50% |
1923+
| Unique messages per sender | 15,360 | 12,720–13,440 |
1924+
| Collision meanings per sender | 0 | 1,920–2,640 |
1925+
| Message entropy | 13.91 bits | 13.54–13.64 bits |
1926+
| Development gate | pass | **fail** |
1927+
1928+
The positive control passed every gate at 100%. ECP7-B24-I sets new ECP-7
1929+
development bests for mean train, mean validation and universal-translator
1930+
validation. It still fails both train thresholds and sender injectivity, while
1931+
worst-link validation is below Batch 21. The joint gate fails and confirmatory
1932+
access remains unauthorized.
1933+
1934+
All 101 optimization records through step 20,000 match Batch 23 and Batch 22
1935+
exactly across their shared fields. Every history record confirms the
1936+
registered routes: sender-only replay through step 20,000 and receiver-only
1937+
replay from the next boundary at step 20,200. The all-link pool follows the
1938+
common trajectory from 1,745 to a
1939+
minimum of 1,414 at step 18,200 and 1,579 at step 20,000.
1940+
1941+
The first receiver-only boundary still observes 1,984 all-link failures, because
1942+
pool mining precedes that boundary's update. Unlike joint B23, subsequent
1943+
receiver-only updates reduce that pool again: it contains 1,990 meanings at
1944+
step 22,000, 1,690 at the selected step 29,400 and 1,710 at the final step
1945+
30,000. The selected checkpoint is at the end of the available horizon rather
1946+
than an early-stopping plateau.
1947+
1948+
The selected diagnostic finds 1,690 meanings failed by all links and 3,142
1949+
failed by at least one link. Mean failed links among those meanings is 11.18.
1950+
This is close to Batch 22's sender-only pattern of 1,513 shared and 3,197
1951+
any-link failures, but the receiver catch-up phase raises mean train by 1.01
1952+
percentage points and mean validation by 0.98 points relative to B22. It does
1953+
not repair the fragmented worst-link bottleneck.
1954+
1955+
Compared with Batch 21, B24 improves mean train by 0.98 percentage points, mean
1956+
validation by 0.85 points and translator validation by 0.90 points. Worst-link
1957+
train falls by 0.23 points, worst-link validation by 0.39 points, and sender
1958+
agreement by 1.46 points. Receiver-only replay therefore improves the average
1959+
code-reader fit while increasing population imbalance.
1960+
1961+
Selected sender codebooks contain 13,214, 12,955, 12,720 and 13,440 unique
1962+
messages, with 2,146, 2,405, 2,640 and 1,920 collision meanings. Population
1963+
validation factor accuracies were `[99.98%, 99.87%, 98.44%, 86.17%]`;
1964+
universal-translator factor accuracies were
1965+
`[100%, 100%, 98.44%, 86.11%]`. Texture improves, while the fixed size
1966+
collision pattern and sender-specific texture differences remain.
1967+
1968+
Both sealed analyses report 65 matching artifact hashes, 16,384 validation-only
1969+
episodes, no confirmatory-test keys, valid local alphabets, and exactly 14
1970+
declared bits for every logged message.
1971+
1972+
## Batch 24 decision
1973+
1974+
ECP7-B24-I is a valid negative result and the strongest mean-performance ECP-7
1975+
development variant so far. Receiver-only catch-up is better than joint replay
1976+
for mean reconstruction, but it does not solve injectivity or worst-link
1977+
generalization. The confirmatory split remains sealed.
1978+
1979+
The selected checkpoint occurs at step 29,400 and training reaches the 30,000
1980+
step cap before the registered 5,000-step patience can evaluate a late plateau.
1981+
A future ECP7-B25 may inherit Batch 24 unchanged and extend only `max_steps` to
1982+
45,000. This isolates whether the receiver-only phase is still improving or
1983+
merely approaching a non-injective plateau. No coefficient, route, predicate,
1984+
architecture, optimizer or data setting may change with that horizon test.

‎docs/research-design-ecp7.md‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1013,3 +1013,43 @@ sealed-test requirement fails. No alternative switch step, gradient route,
10131013
predicate, coefficient, replay batch, refresh, architecture, base loss,
10141014
optimizer, duration, data, translator or seed may be tried inside Batch 23.
10151015
Failure is recorded without confirmatory test access.
1016+
1017+
## Batch 24 preregistration — Receiver-only replay catch-up
1018+
1019+
Status: **preregistered for sealed development**<br>
1020+
Preregistered: July 18, 2026 at 17:09:59 UTC<br>
1021+
Configuration: `config/ecp7-b24-development.yaml`<br>
1022+
Raw configuration SHA-256: `27e737c1eaa3c191253469fb863171c9875e1e9433c2b00b7a578e15609155dc`<br>
1023+
Effective configuration SHA-256: `a01f184249b32859074909039ba86d495854a3c5f063c19f5519e2739135cb9e`
1024+
1025+
Batch 22's sender-only replay separated population-shared failures but created
1026+
link-specific fragmentation. Batch 23 restored joint replay after warmup and
1027+
immediately recreated shared failures. Batch 24 isolates the remaining routing
1028+
hypothesis: let receivers catch up to the separated codes without allowing the
1029+
additional replay loss to move sender codes again.
1030+
1031+
ECP7-B24-I inherits the complete Batch 23 implementation, Batch 21 predicate,
1032+
Batch 20 replay mechanism, Batch 15 architecture, ordinary task batch,
1033+
utilization objective, optimizer, data, temperature schedule, coefficient,
1034+
replay batch, sampler, refresh, warmup, 30,000-step horizon, selection,
1035+
translator and thresholds. The sole change is:
1036+
1037+
- after step 20,000, the additional hard-replay branch detaches sender messages,
1038+
making that branch receiver-only instead of joint.
1039+
1040+
Replay remains sender-only through step 20,000. The ordinary base task updates
1041+
both senders and receivers at every step. Mining still uses hard training
1042+
predictions only. The complete optimization trajectory through step 20,000
1043+
must match Batch 23 and Batch 22 exactly.
1044+
1045+
The batch contains exactly two seed-11 runs:
1046+
1047+
1. the unchanged ECP-6 positive control;
1048+
2. ECP7-B24-I with sender-only then receiver-only hard replay.
1049+
1050+
All existing validity and development gates remain unchanged. The intervention
1051+
fails if any train, validation, translator, injectivity, channel-integrity or
1052+
sealed-test requirement fails. No alternative switch step, gradient route,
1053+
predicate, coefficient, replay batch, refresh, architecture, base loss,
1054+
optimizer, duration, data, translator or seed may be tried inside Batch 24.
1055+
Failure is recorded without confirmatory test access.

‎evidence/ecp7-development/README.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -130,6 +130,12 @@ reached 2,085 at selection. The any-link pool improved to 2,660, but validation
130130
remained 83.50% and injectivity still failed. Exact identities and hashes are in
131131
`batch23-manifest.json`.
132132

133+
Batch 24 changed the second replay phase from joint to receiver-only. It reached
134+
new-best means of 84.69% train, 84.48% validation and 84.55% translator
135+
validation. The selected error pool contains 1,690 shared failures and 3,142
136+
any-link failures, so worst-link accuracy and injectivity still fail. Exact
137+
identities and hashes are in `batch24-manifest.json`.
138+
133139
Reproduce the development arms with:
134140

135141
```bash

0 commit comments

Comments
 (0)