Skip to content

Commit 6e5a9fd

Browse files
authored
Record ECP-7 Batch 16 late-minimax result (#17)
Add the delayed factor-minimax schedule and record the sealed Batch 16 negative result with verified evidence.
1 parent a021707 commit 6e5a9fd

12 files changed

Lines changed: 337 additions & 38 deletions

File tree

‎README.md‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -26,14 +26,15 @@ The protocol uses no words or alphabet, but four meaning-free local symbols. For
2626
See [`docs/results-ecp6.md`](docs/results-ecp6.md) for the conclusion, [`docs/protocol-specification-ecp6.md`](docs/protocol-specification-ecp6.md) for the wire format and [`evidence/ecp6/report.md`](evidence/ecp6/report.md) for the compact confirmatory evidence.
2727

2828
ECP-7 development now tests how much of that result depends on the explicit
29-
one-factor-per-slot architecture. Its first fifteen sealed batches are valid
30-
development results but none passes the full gate. Batch 15 is now strongest:
29+
one-factor-per-slot architecture. Its first sixteen sealed batches are valid
30+
development results but none passes the full gate. Batch 15 remains strongest:
3131
the position-aware design at 30,000 optimization steps reaches 83.46% train
3232
exactness, 82.59% validation and 83.37% translator validation while using
3333
12,585–13,200 hard messages per sender. Validation and translator thresholds pass
34-
together, but the registered train and injectivity thresholds do not. The ECP-6
35-
positive controls remain perfect and the ECP-7 confirmatory test remains sealed. See
36-
[`docs/development-log-ecp7.md`](docs/development-log-ecp7.md).
34+
together, but the registered train and injectivity thresholds do not. Batch 16
35+
added late worst-factor pressure and regressed to 76.46% validation. The ECP-6
36+
positive controls remain perfect and the ECP-7 confirmatory test remains sealed.
37+
See [`docs/development-log-ecp7.md`](docs/development-log-ecp7.md).
3738

3839
## Build on this work
3940

@@ -88,6 +89,7 @@ Do not translate those artifacts in place. New experiment configurations and all
8889
- [`config/ecp7-b13-development.yaml`](config/ecp7-b13-development.yaml) — one additional generic shared sender layer on the Batch 10 base.
8990
- [`config/ecp7-b14-development.yaml`](config/ecp7-b14-development.yaml) — late learning-rate decay on the Batch 10 base.
9091
- [`config/ecp7-b15-development.yaml`](config/ecp7-b15-development.yaml) — the Batch 10 design with a 30,000-step population horizon.
92+
- [`config/ecp7-b16-development.yaml`](config/ecp7-b16-development.yaml) — late normalized factor-minimax pressure on the Batch 15 base.
9193
- [`config/ecp7-positive-control-development.yaml`](config/ecp7-positive-control-development.yaml) — ECP-6 positive control on the ECP-7 split.
9294
- [`config/ecp0.yaml`](config/ecp0.yaml) — machine-readable configuration from step 1.
9395
- [`config/ecp1.yaml`](config/ecp1.yaml) — frozen configuration of the population trial.

‎config/ecp7-b16-development.yaml‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
extends: ecp7-b15-development.yaml
2+
3+
experiment:
4+
title: ECP-7 late normalized factor-minimax refinement
5+
arm: late_normalized_factor_minimax
6+
objective: >-
7+
Test whether emphasizing the worst normalized reconstruction factor only
8+
after the high-entropy Batch 15 protocol is established resolves residual
9+
collisions without changing the architecture or base optimization path.
10+
11+
training:
12+
factor_minimax:
13+
enabled: true
14+
weight: 1.0
15+
start_step: 15000
16+
warmup_steps: 5000

‎docs/AI_AGENT_START.md‎

Lines changed: 22 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ python3.12 -m venv .venv
4141
.venv/bin/ecp6 --config config/ecp6.yaml validate
4242
```
4343

44-
Expected baseline: 89 passing tests and split sizes `14336/1024/1024`.
44+
Expected baseline: 93 passing tests and split sizes `14336/1024/1024`.
4545

4646
## 4. Reproduce ECP-6
4747

@@ -71,7 +71,7 @@ This reruns a known experiment. It must not be presented as a new independent co
7171

7272
## 5. Continue ECP-7 scientifically
7373

74-
Do not modify ECP-0 through ECP-6 or the completed ECP-7 Batch 1 through Batch 15
74+
Do not modify ECP-0 through ECP-6 or the completed ECP-7 Batch 1 through Batch 16
7575
configs. ECP7-B1-I collapsed to 130–139 hard messages. ECP7-B2-I improved its
7676
soft objective but collapsed further to 85–104 hard messages. ECP7-B3-I applied
7777
that loss to straight-through hard messages and improved to 585–972 messages,
@@ -103,12 +103,15 @@ rate and regressed to 79.13% train, 65.49% validation and 71.02% translator
103103
validation. ECP7-B15-I extended constant-rate training to 30,000 steps and is
104104
now strongest at 83.46% train, 82.59% validation and 83.37% translator
105105
validation. Its validation and translator gates pass, but train and injectivity
106-
do not. The confirmatory ECP-7 test is still sealed.
106+
do not. ECP7-B16-I added late normalized factor-minimax pressure. It slightly
107+
improved texture but disrupted color, regressing to 82.80% train, 76.46%
108+
validation and 77.76% translator validation without solving injectivity. Batch
109+
15 remains strongest, and the confirmatory ECP-7 test is still sealed.
107110

108111
Continue with this sequence:
109112

110113
1. Read `docs/research-design-ecp7.md` and `docs/development-log-ecp7.md`.
111-
2. Define exactly one ECP7-B16 intervention and its failure criterion.
114+
2. Define exactly one ECP7-B17 intervention and its failure criterion.
112115
3. Register the variant and immutable configuration hashes before training.
113116
4. Add tests for every new invariant.
114117
5. Use only `smoke` and `develop`; keep the ECP-7 test split sealed.
@@ -121,23 +124,27 @@ Continue with this sequence:
121124

122125
## 6. Recommended next experiment
123126

124-
Batch 15 is the strongest weak-structure result. It passes validation and
125-
translator gates, with residual known-meaning error concentrated in size after
126-
the high-entropy code is established. Earlier Batch 6 minimax pressure started
127-
from step zero on a much weaker protocol. The most informative next step is one
128-
late worst-factor schedule. A clean ECP7-B16 progression is:
127+
Batch 15 remains the strongest weak-structure result. Batch 16 shows that late
128+
worst-factor pressure improves texture slightly but harms color generalization
129+
and does not resolve collisions. The next clean question is whether the global
130+
collisions can be made trainable without factor-to-slot supervision. A clean
131+
ECP7-B17 progression is:
129132

130133
- keep the same world and bit budget;
131134
- keep the complete B15 architecture, horizon and base losses;
132-
- keep normalized factor-minimax weight `0` through step 15,000;
133-
- warm only that weight linearly to `1.0` over steps 15,000–20,000, then hold;
135+
- mine complete-message collision pairs from each sender's training-only
136+
codebook after step 15,000 and refresh them only at registered evaluation
137+
boundaries;
138+
- apply one relaxed full-message collision-probability loss to those mined
139+
pairs, with a preregistered deterministic sampler and coefficient schedule;
134140
- measure injectivity, validation composition and new-reader induction;
135141
- rerun the ECP-6 architecture as the frozen positive control.
136142

137-
Do not change architecture, utilization, optimizer, data, temperature, duration,
138-
validation cadence or development thresholds, and do not combine the late
139-
minimax schedule with variable-length messages or negotiation. That would make
140-
its effect uninterpretable.
143+
Use training meanings only for pair mining. Do not use validation to choose
144+
pairs, and do not change architecture, base utilization, optimizer, data,
145+
temperature, duration, validation cadence or development thresholds. Do not
146+
combine collision replay with factor reweighting, variable-length messages or
147+
negotiation.
141148

142149
## 7. Definition of done for any contribution
143150

‎docs/development-log-ecp7.md‎

Lines changed: 102 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -12,8 +12,18 @@ this log.
1212
| ECP7-B2-I | Intervention | B1 sender plus factor-agnostic code utilization | Generic sequence encoder | completed; gate failed |
1313
| ECP7-B3-I | Intervention | B2 loss on straight-through hard messages | Generic sequence encoder | completed; gate failed |
1414
| ECP7-B4-I | Intervention | B3 plus direct full-message collision loss | Generic sequence encoder | completed; gate failed |
15-
16-
No other ECP-7 variant has been trained.
15+
| ECP7-B5-I | Intervention | B3 plus straight-through sender consensus | Generic sequence encoder | completed; gate failed |
16+
| ECP7-B6-I | Intervention | B3 plus normalized factor minimax | Generic sequence encoder | completed; gate failed |
17+
| ECP7-B7-I | Intervention | Bounded parallel joint-context sender | Generic sequence encoder | completed; gate failed |
18+
| ECP7-B8-I | Intervention | B7 plus algebraic context invariance | Generic sequence encoder | completed; gate failed |
19+
| ECP7-B9-I | Intervention | B7 sender and position-aware MLP decoder | Position-aware MLP | completed; gate failed |
20+
| ECP7-B10-I | Intervention | B9 with 15,000-step optimization | Position-aware MLP | completed; gate failed |
21+
| ECP7-B11-I | Intervention | B10 with late utilization decay | Position-aware MLP | completed; gate failed |
22+
| ECP7-B12-I | Intervention | B10 with deeper shared decoder | Deep position-aware MLP | completed; gate failed |
23+
| ECP7-B13-I | Intervention | B10 with deeper shared sender | Position-aware MLP | completed; gate failed |
24+
| ECP7-B14-I | Intervention | B10 with late learning-rate decay | Position-aware MLP | completed; gate failed |
25+
| ECP7-B15-I | Intervention | B10 with 30,000-step optimization | Position-aware MLP | completed; gate failed |
26+
| ECP7-B16-I | Intervention | B15 with late normalized factor minimax | Position-aware MLP | completed; gate failed |
1727

1828
## Batch 1 preregistration
1929

@@ -689,7 +699,7 @@ Sender agreement rose from 89.51% to 92.31%, and collisions fell substantially.
689699
Population validation factor accuracies were
690700
`[87.24%, 99.93%, 98.44%, 86.18%]`; universal-translator factor accuracies were
691701
`[94.41%, 100%, 97.73%, 81.69%]`. Shape became essentially perfect while color
692-
and size remained the residual errors.
702+
and texture remained the residual errors.
693703

694704
The selected checkpoint was again the final registered step, now 15,000. Its
695705
task loss was `0.0765` and its full-weight code-utilization term was `-0.7548`.
@@ -763,7 +773,7 @@ measure in the wrong direction.
763773

764774
Population validation factor accuracies were
765775
`[87.21%, 99.96%, 98.54%, 85.45%]`; universal-translator factor accuracies were
766-
`[87.50%, 100%, 98.54%, 85.28%]`. Color and size remain the residual errors.
776+
`[87.50%, 100%, 98.54%, 85.28%]`. Color and texture remain the residual errors.
767777

768778
The selected checkpoint was step 14,800. Its task loss was `0.0805`, utilization
769779
loss was `-0.7275`, and scheduled utilization weight was `0.118`. Every setting
@@ -781,8 +791,8 @@ full late utilization pressure was the primary B10 bottleneck. B10 remains the
781791
strongest weak-structure base, and the confirmatory split remains sealed.
782792

783793
The residual error now appears decoder-limited rather than loss-weight-limited:
784-
B10's shared one-hidden-layer decoder plateaued on color and size, and its fresh
785-
translator showed the same size weakness. A future ECP7-B12 may return to the
794+
B10's shared one-hidden-layer decoder plateaued on color and texture, and its
795+
fresh translator showed the same texture weakness. A future ECP7-B12 may return to the
786796
complete B10 schedule and add exactly one second generic shared hidden layer to
787797
both the position-aware receiver and isolated translator. It must preserve the
788798
same flattened positional input and must not add factor-specific paths, binding,
@@ -1043,8 +1053,8 @@ principal gate failure.
10431053
Population validation factor accuracies were
10441054
`[98.86%, 99.75%, 98.44%, 85.05%]`; universal-translator factor accuracies were
10451055
`[100%, 100%, 98.44%, 84.94%]`. Color and shape became essentially perfect.
1046-
Size is now the dominant residual factor, with the fixed texture error also
1047-
preventing exact reconstruction.
1056+
Texture is now the dominant residual factor. The fixed 98.44% size accuracy
1057+
adds a smaller exact-match ceiling.
10481058

10491059
The selected checkpoint was step 28,800 with task loss `0.0675` and utilization
10501060
loss `-0.7399`. Validation first crossed 80% between steps 24,000 and 26,000 and
@@ -1062,9 +1072,92 @@ strongest weak-structure base. It passes composition and new-reader induction;
10621072
only known-meaning exactness and injectivity remain below their registered
10631073
thresholds. The confirmatory split remains sealed.
10641074

1065-
The residual error is concentrated in size after the shared code is already
1075+
The residual error is concentrated in texture after the shared code is already
10661076
established. A future ECP7-B16 may keep the complete B15 design and introduce
10671077
only late normalized worst-factor emphasis: factor-minimax weight `0` through
10681078
step 15,000, linearly warm to `1.0` over 5,000 steps, then hold. This revisits the
10691079
Batch 6 objective only after the high-entropy B15 protocol exists and must not
10701080
change architecture, utilization, duration, data, translator or thresholds.
1081+
1082+
## Batch 16 preregistration
1083+
1084+
Preregistered: July 18, 2026 at 14:07:11 UTC<br>
1085+
Seed: `11`<br>
1086+
Maximum population steps: `30,000`<br>
1087+
Minimum selection steps: `15,000`<br>
1088+
Early-stopping patience: `5,000` steps<br>
1089+
Factor-minimax weight: `0` through step `15,000`, linear to `1.0` at step `20,000`, then hold<br>
1090+
Raw configuration SHA-256: `caef16ec93d0a9424fd3c4af46a987aad63118f24e57e77e9d51cc2eb77e073b`<br>
1091+
Effective configuration SHA-256: `b40f35ab2f03fa8450805908713b4b9a59bbc060c38a4e68e4e3c5be2d2b4efb`<br>
1092+
Split-SHA-256: `4947058c75ab07cb43a87eb82776b12cb2a7e2eeba7114de110d3b852cbc64cd`<br>
1093+
Test unsealed: **no**
1094+
1095+
ECP7-B16-I keeps every Batch 15 architecture, base loss, optimizer, data,
1096+
temperature, duration, selection and translator setting. It adds only the
1097+
existing normalized factor-minimax loss with a delayed coefficient: no minimax
1098+
gradient through step 15,000, a 5,000-step linear warmup, then full weight.
1099+
The unchanged positive control is rerun. No alternative schedule or accompanying
1100+
intervention is admitted into this batch, and smoke remains mechanical only.
1101+
1102+
## Batch 16 results
1103+
1104+
Positive-control run: `runs/ecp7-batch16-control-development/20260718T140854Z-ecp7-development`<br>
1105+
Intervention run: `runs/ecp7-batch16-intervention-development/20260718T140854Z-ecp7-development`<br>
1106+
Test unsealed: **no**
1107+
1108+
| Metric | Positive control | ECP7-B16-I |
1109+
|---|---:|---:|
1110+
| Population train, mean | 100% | 82.8033% |
1111+
| Population train, worst link | 100% | 81.4104% |
1112+
| Population validation, mean | 100% | 76.4648% |
1113+
| Population validation, worst link | 100% | 74.5117% |
1114+
| Universal translator, validation | 100% | 77.7588% |
1115+
| Exact sender-message agreement | 100% | 92.59% |
1116+
| Unique messages per sender | 15,360 | 12,487–13,044 |
1117+
| Collision meanings per sender | 0 | 2,316–2,873 |
1118+
| Message entropy | 13.91 bits | 13.50–13.59 bits |
1119+
| Development gate | pass | **fail** |
1120+
1121+
The positive control passed every gate at 100%. ECP7-B16-I retained a passing
1122+
universal-translator score, but failed known-meaning train exactness, validation
1123+
and injectivity. It therefore remains a negative sealed-development result and
1124+
does not authorize confirmatory access.
1125+
1126+
The trajectory through step 15,000 matches B15 exactly, including task loss
1127+
`0.076486`, train exactness `82.0836%` and validation `72.9797%`. This verifies
1128+
that the delayed schedule isolated the intended late intervention. Minimax
1129+
weight then rose to `0.2` at step 16,000, `0.6` at step 18,000 and `1.0` at step
1130+
20,000.
1131+
1132+
At the selected step 28,000 checkpoint, the minimax loss was `0.11384` at full
1133+
weight, task loss was `0.07254`, and utilization loss was `-0.75143`. Relative
1134+
to B15, validation fell from 82.5867% to 76.4648% and translator validation fell
1135+
from 83.3740% to 77.7588%. The intervention did not reduce collision meanings or
1136+
make any sender injective.
1137+
1138+
Population validation factor accuracies were
1139+
`[90.74%, 99.90%, 98.44%, 86.34%]`; universal-translator factor accuracies were
1140+
`[93.65%, 100%, 98.44%, 84.62%]`. Late minimax slightly improved the weakest
1141+
texture score relative to B15 but disrupted the previously near-perfect color
1142+
generalization. The largest collision bucket also increased from three meanings
1143+
in B15 to four meanings in B16.
1144+
1145+
Both sealed analyses report 65 matching artifact hashes, 16,384 validation-only
1146+
episodes, no confirmatory-test keys, valid local alphabets, and exactly 14
1147+
declared bits for every logged message.
1148+
1149+
## Batch 16 decision
1150+
1151+
ECP7-B16-I is rejected. Generic worst-factor pressure after protocol formation
1152+
trades texture improvement for a larger color regression and does not solve
1153+
injectivity. Batch 15 remains the strongest weak-structure base, and the
1154+
confirmatory split remains sealed.
1155+
1156+
The next intervention should target the remaining hard-message collisions
1157+
without reweighting semantic factors. A future ECP7-B17 may return to B15 and
1158+
add one late, training-only global collision-pair replay objective. It would
1159+
mine colliding meaning pairs from each sender's complete training codebook,
1160+
then penalize their relaxed full-message collision probability. This directly
1161+
addresses the sparse-minibatch limitation observed in Batch 4 while assigning
1162+
no semantic factor to any channel slot. Architecture, base losses, duration,
1163+
data, translator and thresholds must remain unchanged.

0 commit comments

Comments
 (0)