Skip to content

Commit aee9d65

Browse files
authored
Add ECP-7 Batch 26 coordinate replay study (#27)
Record the sealed Batch 26 coordinate replay result, evidence, validation, and fresh-agent handoff.
1 parent f1714bf commit aee9d65

12 files changed

Lines changed: 457 additions & 28 deletions

File tree

‎README.md‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ The protocol uses no words or alphabet, but four meaning-free local symbols. For
2626
See [`docs/results-ecp6.md`](docs/results-ecp6.md) for the conclusion, [`docs/protocol-specification-ecp6.md`](docs/protocol-specification-ecp6.md) for the wire format and [`evidence/ecp6/report.md`](evidence/ecp6/report.md) for the compact confirmatory evidence.
2727

2828
ECP-7 development now tests how much of that result depends on the explicit
29-
one-factor-per-slot architecture. Its first twenty-five sealed batches are valid
29+
one-factor-per-slot architecture. Its first twenty-six sealed batches are valid
3030
development results but none passes the full gate. Batch 15 established the
3131
strong position-aware base: at 30,000 optimization steps it reaches 83.46% train
3232
exactness, 82.59% validation and 83.37% translator validation while using
@@ -55,7 +55,11 @@ worst-link and injectivity gates still failed.
5555
Batch 25 extended that catch-up to 45,000 steps and improved again to 85.26%
5656
train, 85.24% validation and 85.13% translator validation. Receiver performance
5757
now nearly reaches the mathematical ceiling of the non-injective sender
58-
codebooks, so another horizon-only extension is not justified. See
58+
codebooks, so another horizon-only extension is not justified. Batch 26 added
59+
one sender/receiver route cycle and reached 85.38% validation, but did not add
60+
a single net unique code: shared errors became link-specific errors instead.
61+
The replay-routing family is therefore closed; future work must target sender
62+
capacity directly. See
5963
[`docs/development-log-ecp7.md`](docs/development-log-ecp7.md).
6064

6165
## Build on this work
@@ -121,6 +125,7 @@ Do not translate those artifacts in place. New experiment configurations and all
121125
- [`config/ecp7-b23-development.yaml`](config/ecp7-b23-development.yaml) — Batch 22 sender-only replay through step 20,000 followed by restored joint replay gradients.
122126
- [`config/ecp7-b24-development.yaml`](config/ecp7-b24-development.yaml) — Batch 23 routing with sender messages detached for receiver-only replay after step 20,000.
123127
- [`config/ecp7-b25-development.yaml`](config/ecp7-b25-development.yaml) — Batch 24 receiver-only catch-up with an isolated 45,000-step maximum horizon.
128+
- [`config/ecp7-b26-development.yaml`](config/ecp7-b26-development.yaml) — Batch 25 with one preregistered sender/receiver replay-route cycle.
124129
- [`config/ecp7-positive-control-development.yaml`](config/ecp7-positive-control-development.yaml) — ECP-6 positive control on the ECP-7 split.
125130
- [`config/ecp0.yaml`](config/ecp0.yaml) — machine-readable configuration from step 1.
126131
- [`config/ecp1.yaml`](config/ecp1.yaml) — frozen configuration of the population trial.

‎config/ecp7-b26-development.yaml‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
extends: ecp7-b25-development.yaml
2+
3+
experiment:
4+
title: ECP-7 coordinate-descent hard-replay cycle
5+
arm: coordinate_descent_hard_replay_cycle
6+
objective: >-
7+
Test whether a second sender-only separation pulse against saturated
8+
receivers can break remaining code collisions before a final receiver-only
9+
catch-up phase.
10+
11+
training:
12+
global_hard_meaning_replay:
13+
gradient_route_phases:
14+
- end_step: 20000
15+
sender_message_gradients: true
16+
receiver_parameter_gradients: false
17+
- end_step: 30000
18+
sender_message_gradients: false
19+
receiver_parameter_gradients: true
20+
- end_step: 35000
21+
sender_message_gradients: true
22+
receiver_parameter_gradients: false
23+
- end_step: 45000
24+
sender_message_gradients: false
25+
receiver_parameter_gradients: true

‎docs/AI_AGENT_START.md‎

Lines changed: 24 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ python3.12 -m venv .venv
4141
.venv/bin/ecp6 --config config/ecp6.yaml validate
4242
```
4343

44-
Expected baseline: 122 passing tests and split sizes `14336/1024/1024`.
44+
Expected baseline: 125 passing tests and split sizes `14336/1024/1024`.
4545

4646
## 4. Reproduce ECP-6
4747

@@ -71,7 +71,7 @@ This reruns a known experiment. It must not be presented as a new independent co
7171

7272
## 5. Continue ECP-7 scientifically
7373

74-
Do not modify ECP-0 through ECP-6 or the completed ECP-7 Batch 1 through Batch 25
74+
Do not modify ECP-0 through ECP-6 or the completed ECP-7 Batch 1 through Batch 26
7575
configs. ECP7-B1-I collapsed to 130–139 hard messages. ECP7-B2-I improved its
7676
soft objective but collapsed further to 85–104 hard messages. ECP7-B3-I applied
7777
that loss to straight-through hard messages and improved to 585–972 messages,
@@ -149,11 +149,18 @@ ECP7-B25-I extended only that catch-up horizon to 45,000 steps. It improved to
149149
85.13% translator validation. The worst train link exactly matches its 82.81%
150150
unique-code ceiling, so sender injectivity remains the structural blocker and
151151
the confirmatory ECP-7 test remains sealed.
152+
ECP7-B26-I inserted a second sender-only pulse followed by receiver-only
153+
catch-up. It reached small new bests of 85.37% mean train and 85.38% validation
154+
and translator validation, but its unique-message counts were effectively
155+
unchanged. Shared failures fell while any-link failures rose, closing the
156+
replay-routing family as an error-redistribution mechanism rather than an
157+
injectivity solution. The confirmatory ECP-7 test remains sealed.
152158

153159
Continue with this sequence:
154160

155161
1. Read `docs/research-design-ecp7.md` and `docs/development-log-ecp7.md`.
156-
2. Define exactly one ECP7-B26 intervention and its failure criterion.
162+
2. Define exactly one ECP7-B27 direct sender-capacity intervention and its
163+
failure criterion.
157164
3. Register the variant and immutable configuration hashes before training.
158165
4. Add tests for every new invariant.
159166
5. Use only `smoke` and `develop`; keep the ECP-7 test split sealed.
@@ -166,29 +173,20 @@ Continue with this sequence:
166173

167174
## 6. Recommended next experiment
168175

169-
Batch 25 confirms that extra receiver-only catch-up time improves decoder fit
170-
but leaves the sender collision structure almost unchanged. The observed worst
171-
train link exactly reaches its sender's unique-code ceiling, so another horizon
172-
extension cannot solve the gate. A clean ECP7-B26 coordinate-descent progression
173-
is:
174-
175-
- inherit the complete Batch 25 implementation and 45,000-step ceiling;
176-
- keep sender-only replay through step 20,000 and receiver-only replay through
177-
step 30,000 exactly as in Batch 25;
178-
- route the additional replay branch sender-only again for steps 30,001–35,000,
179-
then restore receiver-only replay from step 35,001;
180-
- reuse the registered 5,000-step sender-separation duration;
181-
- keep ordinary base-task updates to both senders and receivers unchanged;
182-
- keep weight `0.25`, the 64-meaning replay batch, 200-step refresh,
183-
seed-derived sampler, predicate and temperature unchanged;
184-
- continue to mine and replay training meanings only;
185-
- measure shared and any-link pool sizes, train exactness, injectivity,
186-
validation and new-reader induction, and rerun the frozen ECP-6 control.
187-
188-
Do not combine the routing cycle with another predicate, coefficient, graded
189-
sampling, collision pressure, a new architecture, factor-specific weights,
190-
optimizer changes, a horizon change, variable-length messages or
191-
negotiation.
176+
Batch 26 shows that another one-sided replay cycle cannot increase sender code
177+
capacity. Receiver-only phases fit the existing codebook ceiling; sender-only
178+
ordinary-task replay converts shared errors into link-specific errors. Do not
179+
try another route boundary, route cycle or horizon extension.
180+
181+
For ECP7-B27, preregister exactly one direct sender-capacity intervention on the
182+
strongest stable base. A defensible candidate is a late, bounded pulse that
183+
explicitly separates globally mined hard-message collisions while preserving
184+
the ordinary reconstruction task. Reuse an already tested loss and coefficient
185+
where possible, activate it only after a shared unchanged trajectory, and
186+
define success primarily as a reproducible increase in unique messages without
187+
loss of worst-link validation. Do not combine this with a new architecture,
188+
new predicate, optimizer change, factor-specific supervision, variable-length
189+
messages or negotiation in the same batch.
192190

193191
## 7. Definition of done for any contribution
194192

‎docs/development-log-ecp7.md‎

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@ this log.
3333
| ECP7-B23-I | Intervention | B22 sender-only replay warmup followed by restored joint replay gradients | Position-aware MLP | valid negative; joint phase recreates shared errors without improving accuracy |
3434
| ECP7-B24-I | Intervention | B23 sender-only warmup followed by receiver-only replay catch-up | Position-aware MLP | valid negative; new-best means but link-specific errors and injectivity fail |
3535
| ECP7-B25-I | Intervention | B24 receiver-only catch-up with a 45,000-step maximum horizon | Position-aware MLP | valid negative; new-best means reach the non-injective code-cap |
36+
| ECP7-B26-I | Intervention | B25 with a second sender-only pulse and final receiver-only catch-up | Position-aware MLP | valid negative; route cycle changes error sharing but not code capacity |
3637

3738
## Batch 1 preregistration
3839

@@ -2088,3 +2089,98 @@ receiver-only through 30,000, a second sender-only separation pulse through
20882089
registered 5,000-step duration and directly tests whether saturated receivers
20892090
provide a better fixed target for breaking remaining sender collisions. No
20902091
other setting may change.
2092+
2093+
## Batch 26 preregistration
2094+
2095+
Preregistered: July 18, 2026 at 17:46:02 UTC<br>
2096+
Seed: `11`<br>
2097+
Maximum population steps: `45,000`<br>
2098+
Minimum selection steps: `15,000`<br>
2099+
Early-stopping patience: `5,000` steps<br>
2100+
Hard-meaning predicate: all `16` current population links have an incorrect factor<br>
2101+
Replay route through step 20,000: **sender-only**<br>
2102+
Replay route steps 20,001–30,000: **receiver-only**<br>
2103+
Replay route steps 30,001–35,000: **sender-only**<br>
2104+
Replay route steps 35,001–45,000: **receiver-only**<br>
2105+
Ordinary-task sender and receiver gradients: **enabled throughout**<br>
2106+
Replay weight: `0` through step `15,000`, linear to `0.25` at step `20,000`, then hold<br>
2107+
Replay batch size: `64` meanings, uniform with replacement<br>
2108+
Training-only pool refresh interval: `200` steps<br>
2109+
Replay sampler seed offset: `+1211`<br>
2110+
Raw configuration SHA-256: `9721d3af452b72788568f7fc5262056aa770e443c0a3bc35112183a08c2badb7`<br>
2111+
Effective configuration SHA-256: `be1310f79b4b7a72134393292480244eeba123d4828b3c94dbf32787dd6a4e3b`<br>
2112+
Split-SHA-256: `4947058c75ab07cb43a87eb82776b12cb2a7e2eeba7114de110d3b852cbc64cd`<br>
2113+
Test unsealed: **no**
2114+
2115+
ECP7-B26-I inherits Batch 25 and changes only the replay gradient-route
2116+
schedule. The complete trajectory through step 30,000 remains identical. A
2117+
second 5,000-step sender-only pulse then uses the saturated receivers as fixed
2118+
targets, followed by 10,000 steps of receiver-only catch-up. The ordinary task
2119+
remains joint throughout. Exact phase-boundary and one-sided-gradient
2120+
invariants are tested. The positive control is rerun, and no alternative cycle
2121+
or accompanying intervention is admitted into this batch.
2122+
2123+
## Batch 26 results
2124+
2125+
Positive-control run: `runs/ecp7-batch26-control-development/20260718T174733Z-ecp7-development`<br>
2126+
Intervention run: `runs/ecp7-batch26-intervention-development/20260718T174733Z-ecp7-development`<br>
2127+
Test unsealed: **no**
2128+
2129+
| Metric | Positive control | ECP7-B26-I |
2130+
|---|---:|---:|
2131+
| Population train, mean | 100% | **85.3725%** |
2132+
| Population train, worst link | 100% | 82.7427% |
2133+
| Population validation, mean | 100% | **85.3760%** |
2134+
| Population validation, worst link | 100% | 82.7148% |
2135+
| Universal translator, validation | 100% | **85.3760%** |
2136+
| Exact sender-message agreement | 100% | 92.38% |
2137+
| Unique messages per sender | 15,360 | 12,720–13,440 |
2138+
| Collision meanings per sender | 0 | 1,920–2,640 |
2139+
| Message entropy | 13.91 bits | 13.54–13.64 bits |
2140+
| Development gate | pass | **fail** |
2141+
2142+
The positive control passed every gate at 100%. ECP7-B26-I sets small new
2143+
ECP-7 bests for mean train, mean validation, translator validation and sender
2144+
agreement. It still fails both train thresholds and sender injectivity; the
2145+
worst validation link merely ties Batch 25. The joint gate fails and
2146+
confirmatory access remains unauthorized.
2147+
2148+
All 151 optimization records through step 30,000 match Batch 25 exactly. At the
2149+
second sender-only boundary the all-link pool drops from 1,710 at step 30,000
2150+
to 1,264 at step 30,200 and reaches its minimum of 1,209 at step 30,800. The
2151+
selected checkpoint occurs at step 34,200, during this sender-only pulse, with
2152+
1,330 shared failures. The final receiver-only phase begins with 1,648 shared
2153+
failures at step 35,200 and early stopping terminates at step 39,200, 5,000
2154+
steps after selection. Receiver catch-up therefore does not improve on the
2155+
sender-only checkpoint.
2156+
2157+
The selected diagnostic finds 1,330 meanings failed by all links and 3,262
2158+
failed by at least one link, with 10.29 mean failed links per erroneous
2159+
meaning. Relative to Batch 25, the extra sender pulse removes 347 shared
2160+
failures but creates 295 additional any-link failures. This reproduces the
2161+
sender-only tradeoff observed in Batch 22: common errors become link-specific
2162+
rather than disappearing.
2163+
2164+
Selected sender codebooks contain 13,438, 12,960, 12,720 and 13,440 unique
2165+
messages, with 1,922, 2,400, 2,640 and 1,920 collision meanings. Batch 25 used
2166+
13,440, 12,960, 12,720 and 13,440 unique messages. The second sender pulse thus
2167+
adds no code capacity and regresses sender 0 by two meanings. Population and
2168+
universal-translator validation factor accuracies are both
2169+
`[100%, 100%, 98.44%, 86.94%]`. The stable size and texture collisions remain
2170+
the limiting structure.
2171+
2172+
Both sealed analyses report 65 matching artifact hashes, 16,384
2173+
validation-only episodes, no confirmatory-test keys, valid local alphabets,
2174+
and exactly 14 declared bits for every logged message.
2175+
2176+
## Batch 26 decision
2177+
2178+
ECP7-B26-I is a valid negative result. Its very small mean improvements come
2179+
from redistributing reader errors, not from making the sender codebooks more
2180+
injective. Repeating or extending one-sided ordinary-task replay is therefore
2181+
not justified: receiver-only training is already code-capacity-limited, while
2182+
sender-only training reduces shared errors by fragmenting them across links.
2183+
2184+
The ECP-7 confirmatory split remains sealed. Any future batch must change the
2185+
sender collision objective or sender representation directly, rather than add
2186+
another route phase, route boundary or horizon extension.

‎docs/research-design-ecp7.md‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1091,3 +1091,59 @@ sealed-test requirement fails. No alternative horizon, gradient route,
10911091
predicate, coefficient, replay batch, refresh, architecture, base loss,
10921092
optimizer, data, translator or seed may be tried inside Batch 25. Failure is
10931093
recorded without confirmatory test access.
1094+
1095+
## Batch 26 preregistration — Coordinate-descent replay cycle
1096+
1097+
Status: **completed sealed development; valid negative**<br>
1098+
Preregistered: July 18, 2026 at 17:46:02 UTC<br>
1099+
Configuration: `config/ecp7-b26-development.yaml`<br>
1100+
Raw configuration SHA-256: `9721d3af452b72788568f7fc5262056aa770e443c0a3bc35112183a08c2badb7`<br>
1101+
Effective configuration SHA-256: `be1310f79b4b7a72134393292480244eeba123d4828b3c94dbf32787dd6a4e3b`
1102+
1103+
Batch 25 shows that receiver-only catch-up nearly saturates the mathematical
1104+
capacity of the learned sender codebooks. The worst train link exactly matches
1105+
its sender's 82.81% unique-code ceiling. Batch 26 tests whether one additional
1106+
sender-only separation pulse against those saturated receivers can break
1107+
collisions before a final decoder catch-up phase.
1108+
1109+
ECP7-B26-I inherits the complete Batch 25 implementation, 45,000-step ceiling,
1110+
hard-meaning predicate, replay mechanism, architecture, ordinary task batch,
1111+
utilization objective, optimizer, data, temperature schedule, coefficient,
1112+
replay batch, sampler, refresh, warmup, selection, translator and thresholds.
1113+
The sole change is the explicit one-sided replay schedule:
1114+
1115+
1. sender-only through step 20,000;
1116+
2. receiver-only through step 30,000;
1117+
3. sender-only through step 35,000;
1118+
4. receiver-only through step 45,000.
1119+
1120+
Each boundary is inclusive, so the next route begins on the following step.
1121+
The ordinary base task updates both senders and receivers at every step. Mining
1122+
still uses hard training predictions only. The complete optimization trajectory
1123+
through step 30,000 must match Batch 25 and Batch 24 exactly.
1124+
1125+
The batch contains exactly two seed-11 runs:
1126+
1127+
1. the unchanged ECP-6 positive control;
1128+
2. ECP7-B26-I with the coordinate-descent replay cycle.
1129+
1130+
All existing validity and development gates remain unchanged. The intervention
1131+
fails if any train, validation, translator, injectivity, channel-integrity or
1132+
sealed-test requirement fails. No alternative phase boundary, gradient route,
1133+
predicate, coefficient, replay batch, refresh, architecture, base loss,
1134+
optimizer, horizon, data, translator or seed may be tried inside Batch 26.
1135+
Failure is recorded without confirmatory test access.
1136+
1137+
### Batch 26 outcome
1138+
1139+
The registered seed-11 intervention reached 85.37% mean train, 82.74%
1140+
worst-link train, 85.38% mean validation, 82.71% worst-link validation and
1141+
85.38% universal-translator validation. Its sender codebooks used 13,438,
1142+
12,960, 12,720 and 13,440 unique messages, effectively unchanged from Batch
1143+
25. The second sender-only pulse reduced shared errors but increased meanings
1144+
failed by at least one population link. It therefore failed the unchanged
1145+
train and injectivity gates without accessing the confirmatory split.
1146+
1147+
This completes the registered replay-routing family. Further route cycling or
1148+
horizon extension is disallowed as an unsupported repetition; any later
1149+
development batch must preregister a direct sender-capacity intervention.

‎evidence/ecp7-development/README.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -143,6 +143,14 @@ validation, but the observed worst train link exactly reached its sender's
143143
receiver training time. Exact identities and hashes are in
144144
`batch25-manifest.json`.
145145

146+
Batch 26 inserted one second sender-only pulse followed by receiver-only
147+
catch-up. It reached small new-best means of 85.37% train and 85.38% validation
148+
and translator validation, but its unique-message counts were effectively
149+
unchanged. Shared failures fell from 1,677 to 1,330 while any-link failures
150+
rose from 2,967 to 3,262, confirming that route cycling redistributes errors
151+
without fixing sender collisions. Exact identities and hashes are in
152+
`batch26-manifest.json`.
153+
146154
Reproduce the development arms with:
147155

148156
```bash

0 commit comments

Comments
 (0)