Skip to content

Commit 8711e9c

Browse files
committed
Add ECP-7 Batch 28 residual sender study
1 parent d8de32e commit 8711e9c

12 files changed

Lines changed: 365 additions & 18 deletions

File tree

‎README.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,10 @@ Batch 27 directly penalized globally mined sender collisions after step 30,000.
6262
It reduced collision multiplicity and reached 85.38% validation, but unique
6363
code counts again remained unchanged and worst-link validation regressed to
6464
82.13%. Both replay routing and the existing collision-loss schedule are now
65-
closed; future work must change sender representation or code occupancy. See
65+
closed. Batch 28 added an identity-initialized generic residual sender block,
66+
but it collapsed to 9.65% validation and roughly 8,000 unique messages. Generic
67+
depth active from the start promotes memorization rather than compositional
68+
generalization. See
6669
[`docs/development-log-ecp7.md`](docs/development-log-ecp7.md).
6770

6871
## Build on this work
@@ -130,6 +133,7 @@ Do not translate those artifacts in place. New experiment configurations and all
130133
- [`config/ecp7-b25-development.yaml`](config/ecp7-b25-development.yaml) — Batch 24 receiver-only catch-up with an isolated 45,000-step maximum horizon.
131134
- [`config/ecp7-b26-development.yaml`](config/ecp7-b26-development.yaml) — Batch 25 with one preregistered sender/receiver replay-route cycle.
132135
- [`config/ecp7-b27-development.yaml`](config/ecp7-b27-development.yaml) — Batch 25 plus one late bounded direct sender-collision pulse.
136+
- [`config/ecp7-b28-development.yaml`](config/ecp7-b28-development.yaml) — Batch 25 with one zero-initialized shared residual sender interaction.
133137
- [`config/ecp7-positive-control-development.yaml`](config/ecp7-positive-control-development.yaml) — ECP-6 positive control on the ECP-7 split.
134138
- [`config/ecp0.yaml`](config/ecp0.yaml) — machine-readable configuration from step 1.
135139
- [`config/ecp1.yaml`](config/ecp1.yaml) — frozen configuration of the population trial.

‎config/ecp7-b28-development.yaml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
extends: ecp7-b25-development.yaml
2+
3+
experiment:
4+
title: ECP-7 identity-preserving residual parallel sender
5+
arm: identity_preserving_residual_parallel_sender
6+
objective: >-
7+
Test whether one zero-initialized shared residual interaction layer can
8+
break the stable sender code-capacity ceiling while preserving the
9+
successful shallow joint-context path and all meaning-free constraints.
10+
11+
agents:
12+
sender:
13+
family: residual_bounded_parallel_sender

‎docs/AI_AGENT_START.md‎

Lines changed: 23 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ python3.12 -m venv .venv
4141
.venv/bin/ecp6 --config config/ecp6.yaml validate
4242
```
4343

44-
Expected baseline: 127 passing tests and split sizes `14336/1024/1024`.
44+
Expected baseline: 130 passing tests and split sizes `14336/1024/1024`.
4545

4646
## 4. Reproduce ECP-6
4747

@@ -71,7 +71,7 @@ This reruns a known experiment. It must not be presented as a new independent co
7171

7272
## 5. Continue ECP-7 scientifically
7373

74-
Do not modify ECP-0 through ECP-6 or the completed ECP-7 Batch 1 through Batch 27
74+
Do not modify ECP-0 through ECP-6 or the completed ECP-7 Batch 1 through Batch 28
7575
configs. ECP7-B1-I collapsed to 130–139 hard messages. ECP7-B2-I improved its
7676
soft objective but collapsed further to 85–104 hard messages. ECP7-B3-I applied
7777
that loss to straight-through hard messages and improved to 585–972 messages,
@@ -161,11 +161,16 @@ exactly the same 13,440, 12,960, 12,720 and 13,440 unique-message counts as
161161
Batch 25. Worst-link validation regressed to 82.13%, so the existing collision
162162
loss is also closed as a capacity solution. The confirmatory test remains
163163
sealed.
164+
ECP7-B28-I added one zero-initialized shared residual sender interaction. Tests
165+
prove an identical initial function and unchanged RNG state, but the trainable
166+
branch still collapsed to 53.91% train, 9.65% validation, 11.62% translator
167+
validation and only 7,928–8,333 messages. Generic sender depth active from the
168+
first update is therefore rejected. The confirmatory test remains sealed.
164169

165170
Continue with this sequence:
166171

167172
1. Read `docs/research-design-ecp7.md` and `docs/development-log-ecp7.md`.
168-
2. Define exactly one ECP7-B28 sender-representation or occupancy intervention and its
173+
2. Define at most one ECP7-B29 late-activation residual intervention and its
169174
failure criterion.
170175
3. Register the variant and immutable configuration hashes before training.
171176
4. Add tests for every new invariant.
@@ -179,20 +184,21 @@ Continue with this sequence:
179184

180185
## 6. Recommended next experiment
181186

182-
Batch 27 shows that the existing relaxed collision-pair loss can reduce
183-
collision multiplicity but cannot create newly occupied hard codes. Do not try
184-
another coefficient, pulse boundary, route cycle or horizon extension.
185-
186-
For ECP7-B28, preregister exactly one factor-agnostic sender representation or
187-
genuinely different occupancy-creation intervention. A defensible
188-
representation candidate is an identity-preserving residual interaction block
189-
on the successful bounded-parallel sender: retain the complete shallow joint
190-
context and slot heads, add one shared nonlinear residual branch initialized to
191-
zero output, and change no receiver, loss, data or training schedule. This
192-
isolates whether missing joint-context interaction capacity caused the stable
193-
code ceiling while avoiding Batch 13's replacement of the successful shallow
194-
path. Do not combine it with collision replay, a new coefficient, factor-slot
195-
bindings, optimizer changes, variable-length messages or negotiation.
187+
Batch 28 shows that identity initialization alone does not protect the shallow
188+
compositional trajectory: a residual branch trainable from step 1 rapidly
189+
becomes dominant and memorizes the training relation. Do not try another
190+
always-active generic layer, width or activation.
191+
192+
At most one ECP7-B29 residual follow-up is justified: inherit B28's exact
193+
zero-RNG representation and B25's complete training setup, freeze the residual
194+
weight and bias at zero through step 30,000, then activate only those parameters
195+
for the remaining registered horizon. The purpose is to isolate whether late
196+
representation capacity can split an already established code without
197+
relearning the compositional protocol. Predefine one activation boundary and
198+
do not combine it with collision replay, gradient-route changes, a new loss,
199+
factor-slot bindings, optimizer changes, variable-length messages or
200+
negotiation. If this also fails, close ECP-7 development rather than continue a
201+
residual schedule search.
196202

197203
## 7. Definition of done for any contribution
198204

‎docs/development-log-ecp7.md‎

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,7 @@ this log.
3535
| ECP7-B25-I | Intervention | B24 receiver-only catch-up with a 45,000-step maximum horizon | Position-aware MLP | valid negative; new-best means reach the non-injective code-cap |
3636
| ECP7-B26-I | Intervention | B25 with a second sender-only pulse and final receiver-only catch-up | Position-aware MLP | valid negative; route cycle changes error sharing but not code capacity |
3737
| ECP7-B27-I | Intervention | B25 plus one late bounded direct sender-collision pulse | Position-aware MLP | valid negative; collision multiplicity falls but unique-code capacity is unchanged |
38+
| ECP7-B28-I | Intervention | B25 with one zero-initialized shared sender residual interaction | Position-aware MLP | valid negative; generic residual depth drives memorization and code collapse |
3839

3940
## Batch 1 preregistration
4041

@@ -2288,3 +2289,107 @@ The ECP-7 confirmatory split remains sealed. A future batch must change the
22882289
factor-agnostic sender representation or introduce a genuinely different
22892290
occupancy-creation mechanism; it may not combine that change with another
22902291
replay-route or collision-coefficient search.
2292+
2293+
## Batch 28 preregistration
2294+
2295+
Preregistered: July 18, 2026 at 18:30:20 UTC<br>
2296+
Seed: `11`<br>
2297+
Maximum population steps: `45,000`<br>
2298+
Minimum selection steps: `15,000`<br>
2299+
Early-stopping patience: `5,000` steps<br>
2300+
Base architecture: B25 bounded-parallel sender and position-aware MLP receiver<br>
2301+
Representation addition: one shared hidden-to-hidden residual projection<br>
2302+
Residual activation: `tanh`<br>
2303+
Residual weight and bias initialization: exact zero<br>
2304+
Factor-specific sender paths or bindings: **none**<br>
2305+
Additional collision loss: **none**<br>
2306+
Raw configuration SHA-256: `1ce3094d0e372f4b052773a2c44d1abe36cd6255aea96f4218c175e5f93977d7`<br>
2307+
Effective configuration SHA-256: `7bb7c2c185ed46f580464fe4364b8070bf5fdb071d3d64b16ec66b7fbe7ec307`<br>
2308+
Split-SHA-256: `4947058c75ab07cb43a87eb82776b12cb2a7e2eeba7114de110d3b852cbc64cd`<br>
2309+
Test unsealed: **no**
2310+
2311+
ECP7-B28-I inherits Batch 25 and changes only the sender family. The successful
2312+
shallow joint context and four capacity-bounded slot heads remain intact. A
2313+
single shared projection learns an additive `tanh` correction to that context.
2314+
Its parameters are constructed directly as zeros, so the new branch contributes
2315+
exactly zero and consumes no random draws at initialization. Under the same
2316+
seed, every pre-existing sender parameter and all later agent initialization
2317+
therefore remain identical to the B25 base before optimization begins.
2318+
2319+
The positive control is rerun. All existing validity and development gates
2320+
remain unchanged. The representation hypothesis also requires a selected
2321+
minimum sender codebook larger than Batch 25's 12,720-message minimum without a
2322+
lower worst-link validation score. No collision replay from Batch 27, new loss,
2323+
residual width, activation, initialization, receiver change, route change,
2324+
optimizer, horizon, data, translator or seed may be tried inside Batch 28.
2325+
Failure is recorded without confirmatory test access.
2326+
2327+
## Batch 28 results
2328+
2329+
Positive-control run: `runs/ecp7-batch28-control-development/20260718T183206Z-ecp7-development`<br>
2330+
Intervention run: `runs/ecp7-batch28-intervention-development/20260718T183329Z-ecp7-development`<br>
2331+
Test unsealed: **no**
2332+
2333+
| Metric | Positive control | ECP7-B28-I |
2334+
|---|---:|---:|
2335+
| Population train, mean | 100% | 53.9063% |
2336+
| Population train, worst link | 100% | 53.1180% |
2337+
| Population validation, mean | 100% | 9.6497% |
2338+
| Population validation, worst link | 100% | 8.1055% |
2339+
| Universal translator, validation | 100% | 11.6211% |
2340+
| Exact sender-message agreement | 100% | 83.80% |
2341+
| Unique messages per sender | 15,360 | 7,928–8,333 |
2342+
| Collision meanings per sender | 0 | 7,027–7,432 |
2343+
| Message entropy | 13.91 bits | 12.65–12.76 bits |
2344+
| Development gate | pass | **fail** |
2345+
2346+
The positive control passed every gate at 100%. ECP7-B28-I fails train,
2347+
validation, translator, injectivity and the added representation criterion by
2348+
large margins. Its selected checkpoint occurs at step 17,000; patience stops
2349+
training at step 22,000. The joint gate fails and confirmatory access remains
2350+
unauthorized.
2351+
2352+
Implementation tests prove that the residual sender initially has the same
2353+
pre-existing weights and hard messages as the shallow B25 sender and consumes
2354+
no additional random draws. The first optimization update is nevertheless
2355+
allowed to move the zero residual parameters, so the learned trajectories
2356+
diverge immediately. At step 200 B28 already fits training faster than B25
2357+
(2.35% versus 1.49%), but by step 1,000 its validation is only 0.63% versus
2358+
B25's 1.95%. At step 5,000 B28 reaches 41.69% train and 2.23% validation,
2359+
where B25 reaches 71.27% and 52.39%.
2360+
2361+
This gap keeps widening. At step 15,000 B28 has 53.21% train but only 9.42%
2362+
validation, compared with B25's 82.08% train and 72.98% validation. The
2363+
selected residual weight norms are 23.16–23.39 and bias norms are 1.32–1.49;
2364+
the initially null branch has become a dominant transformation rather than a
2365+
small correction.
2366+
2367+
Selected sender codebooks contain only 7,928, 7,987, 8,333 and 8,150 unique
2368+
messages. The selected diagnostic finds 3,902 meanings failed by all links and
2369+
8,720 failed by at least one link, with 12.12 mean failed links per erroneous
2370+
meaning. The all-link replay pool reaches a minimum of 3,716 at step 18,600 and
2371+
then regrows to 4,139 at the final step.
2372+
2373+
Population validation factor accuracies were
2374+
`[80.33%, 25.42%, 99.85%, 96.20%]`; universal-translator factor accuracies were
2375+
`[80.62%, 26.51%, 99.80%, 96.73%]`. The residual model largely memorizes the
2376+
training relation for the held-out shape dimension instead of preserving the
2377+
shallow model's compositional color-shape generalization.
2378+
2379+
Both sealed analyses report 65 matching artifact hashes, 16,384
2380+
validation-only episodes, no confirmatory-test keys, valid local alphabets,
2381+
and exactly 14 declared bits for every logged message.
2382+
2383+
## Batch 28 decision
2384+
2385+
ECP7-B28-I is a valid negative result. Identity-preserving initialization does
2386+
not preserve the successful optimization path when generic residual capacity
2387+
is trainable from the first update. Like Batch 13's replacement layer, the
2388+
residual layer promotes training-set memorization, code collapse and severe
2389+
compositional-generalization loss.
2390+
2391+
The ECP-7 confirmatory split remains sealed. Generic sender depth from the
2392+
start of training is closed. If one final residual test is attempted, it must
2393+
freeze the exact-zero branch through the already successful B25 trajectory and
2394+
activate only that representation afterward; no architectural, objective or
2395+
route change may accompany it.

‎docs/research-design-ecp7.md‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1205,3 +1205,60 @@ minimum-capacity/worst-link criterion without accessing the confirmatory split.
12051205
This closes coefficient and timing development for the existing relaxed
12061206
collision-pair loss. Any later batch must preregister a distinct sender
12071207
representation or occupancy-creation mechanism.
1208+
1209+
## Batch 28 preregistration — Identity-preserving residual parallel sender
1210+
1211+
Status: **completed sealed development; valid negative**<br>
1212+
Preregistered: July 18, 2026 at 18:30:20 UTC<br>
1213+
Configuration: `config/ecp7-b28-development.yaml`<br>
1214+
Raw configuration SHA-256: `1ce3094d0e372f4b052773a2c44d1abe36cd6255aea96f4218c175e5f93977d7`<br>
1215+
Effective configuration SHA-256: `7bb7c2c185ed46f580464fe4364b8070bf5fdb071d3d64b16ec66b7fbe7ec307`
1216+
1217+
Batch 13 replaced the successful shallow sender context with an additional
1218+
nonlinear layer and collapsed. Batches 25–27 now show a stable hard-code
1219+
capacity ceiling that receiver training, replay routing and direct collision
1220+
pressure do not cross. Batch 28 isolates a representation hypothesis without
1221+
discarding the shallow path: add one identity-preserving residual interaction
1222+
to the shared joint context.
1223+
1224+
ECP7-B28-I inherits the full Batch 25 configuration, including its 45,000-step
1225+
horizon, receiver-only hard-meaning replay after step 20,000, architecture on
1226+
the receiver side, losses, optimizer, data, temperature, selection, translator
1227+
and thresholds. The sole configuration change is the sender family from
1228+
`bounded_parallel_sender` to `residual_bounded_parallel_sender`.
1229+
1230+
The new sender first computes the unchanged shallow context. It then adds
1231+
`tanh(W context + b)` before the unchanged slot heads. `W` and `b` are created
1232+
as exact zero tensors without consuming random draws. Consequently the residual
1233+
sender has exactly the same initial discrete messages and pre-existing
1234+
parameters as the shallow sender under the same seed, while its residual
1235+
parameters receive gradients immediately. The branch is shared by all slots
1236+
and sees only the joint context; it has no factor-local projection, factor-slot
1237+
binding, semantic symbol assignment or capacity change.
1238+
1239+
The batch contains exactly two seed-11 runs:
1240+
1241+
1. the unchanged ECP-6 positive control;
1242+
2. ECP7-B28-I with the residual bounded-parallel sender.
1243+
1244+
All existing validity and development gates remain unchanged. Additionally,
1245+
the representation claim requires the selected minimum unique-message count to
1246+
exceed Batch 25's 12,720 while worst-link validation is at least Batch 25's
1247+
82.7148%. No Batch 27 collision loss, alternative residual width, activation,
1248+
initialization, receiver, replay route, base loss, optimizer, horizon, data,
1249+
translator or seed may be tried inside Batch 28. Failure is recorded without
1250+
confirmatory test access.
1251+
1252+
### Batch 28 outcome
1253+
1254+
The registered seed-11 residual sender reached 53.91% mean train, 53.12%
1255+
worst-link train, 9.65% mean validation, 8.11% worst-link validation and 11.62%
1256+
universal-translator validation. Its sender codebooks collapsed to 7,928–8,333
1257+
unique messages. Although tests prove exact functional and RNG-equivalent
1258+
initialization, the trainable residual rapidly became dominant and destroyed
1259+
the shallow sender's compositional trajectory. It failed every performance and
1260+
injectivity gate without accessing the confirmatory split.
1261+
1262+
This closes generic sender-depth changes active from the start of training. A
1263+
future residual experiment is justified only as a single late-activation test
1264+
that preserves the complete established shallow trajectory before unfreezing.

‎evidence/ecp7-development/README.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -158,6 +158,13 @@ message entropy, but produced exactly the same unique-message counts as Batch
158158
therefore reshapes occupied codes without creating new ones. Exact identities
159159
and hashes are in `batch27-manifest.json`.
160160

161+
Batch 28 added one zero-initialized shared residual interaction to the
162+
bounded-parallel sender. Despite exact functional and RNG-equivalent
163+
initialization, the trainable branch collapsed code use to 7,928–8,333 messages
164+
and validation to 9.65%. Generic sender depth active from the first update
165+
therefore promotes memorization rather than compositional generalization. Exact
166+
identities and hashes are in `batch28-manifest.json`.
167+
161168
Reproduce the development arms with:
162169

163170
```bash

0 commit comments

Comments
 (0)