Skip to content

Commit c818aaa

Browse files
jakemannixclaude
andcommitted
runs: real-corpus de-risked (SQuAD 436 titles, 0 tokenize fails); C2 slow, milestone-3 chained
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent c746253 commit c818aaa

1 file changed

Lines changed: 12 additions & 0 deletions

File tree

‎runs/REPORT.md‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -100,3 +100,15 @@ once C2 frees the GPU:
100100
Stats protocol: fixed seeded held-out bank (distinct entities), single base; the harness reports the
101101
worse-side metrics. **Highest-risk question (per the goal): does a real-text-meta-trained skill recall
102102
held-out real entities, or does the PR≈10 / cos-0.77 Gemma answer-prefix geometry cap it post-whitening.**
103+
104+
### Real-corpus path de-risked on CPU + milestone-3 chained (2026-06-10)
105+
Verified locally with the real Gemma tokenizer (no GPU): bundled + SQuAD episodes tokenize with
106+
**0 round-trip failures**; build_cache now skips any that do (arbitrary real spans). SQuAD: use
107+
`rajpurkar/squad` + `shuffle(seed=0)` → 5000 records span **436 article titles**, held-out-by-title
108+
split = **327 train / 109 eval entities (0 overlap)**, 3883/1117 facts. So the real meta-training
109+
data + held-out eval are ready and meaningful.
110+
- **C2 is slow:** `pressure_whiten` runs ~4× slower (multifact k≤32 → ~1000-token ingests → longer
111+
write-BPTT), ~50 min for 3000 steps. So the tail is sequential GPU: C2 (~45 min more) → milestone-3
112+
(~50 min). Chained on the VM (`/content/chain_m3.sh`, detached): waits for C2, pulls latest, runs
113+
`real_text_sweep.sh`. Keepalive holds the VM; results land in `runs/acc_real_*.json` +
114+
`runs/real_eval_*.json` (+ `runs/chain.log` for progress).

0 commit comments

Comments
 (0)