Skip to content

Commit 8238ddb

Browse files
committed
Record end-to-end test of adaptive 2464 resolution: not adopted
Regenerated 42/aysaekanger/isporeu at 2464+rope-rebuild+8192 tokens and scored end-to-end. 2464 eliminates the isporeu runaway (reads the dense table cleanly, KBA x4109 -> x0) and lifts 42 extraction rField 0.76 -> 0.99, but flips no poster to a pass (42 lands 0.01 short; isporeu word regresses as the 8B structures the richer text into fewer matched sections). Per-poster gate kept nothing; caches stay at 1540. Resolution is a real mechanical lever but not the end-to-end lever here -- remaining failures are 8B-structuring / GT-boundary, not OCR starvation.
1 parent 5a7fa46 commit 8238ddb

1 file changed

Lines changed: 30 additions & 0 deletions

File tree

calibration/vlm/RESOLUTION.md

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -98,3 +98,33 @@ better than xy_cut). The resolution ceiling is liftable and helps the worst
9898
posters, but not for free, so the next experiment is adaptive resolution + token
9999
budget, not a bigger fixed render. Settings and model-shopping are dead ends:
100100
the config is already optimal and there is nothing newer to buy.
101+
102+
## 5. End-to-end test of adaptive 2464 (2026-07-28): mechanically sound, no pass gains
103+
104+
Ran the adaptive-resolution follow-up end-to-end on the three failing starved
105+
posters the extraction-stage data most favored: 42, aysaekanger, isporeu,
106+
regenerated at `--longest 2464 --rope-rebuild --max-new-tokens 8192`, then scored
107+
through the full pipeline (scrub + best-of 8B).
108+
109+
Result: it flips NONE of them to a pass.
110+
111+
poster 1540 (w/r/n/f) 2464 (w/r/n/f) verdict
112+
42 0.73/0.71/0.80/0.74 ❌ 0.83/0.74/0.94/0.71 ❌ +extraction, 0.01 short on rougeL
113+
aysaekanger 0.81/0.60/1.00/1.53 ❌ 0.87/0.58/0.83/0.89 ❌ f fixed, rougeL -0.02
114+
isporeu 0.62/0.67/0.33/0.78 ❌ 0.53/0.66/0.43/0.73 ❌ word regressed, rougeL -0.01
115+
116+
Two real findings, neither a win:
117+
- 2464 ELIMINATES the isporeu runaway: at 1540 it loops to the 16384 token cap
118+
(KBA x4109); at 2464 it reads the dense cost-effectiveness table cleanly in
119+
4161 tokens, KBA x0. Resolution, not decoding, is what tames that loop -- but
120+
the 8B then structures the richer text into FEWER matched sections (word
121+
0.62 -> 0.53), so end-to-end it does not improve.
122+
- 42's extraction rField (0.76 -> 0.99 at 2464) does lift its end-to-end rougeL
123+
0.71 -> 0.74, but the 8B stage caps it 0.01 below the 0.75 pass line. 42's
124+
bottleneck is the 8B structuring, not OCR resolution.
125+
126+
Decision: NOT adopted. The per-poster gate kept nothing (no poster both improves
127+
rougeL AND changes outcome enough to justify a mixed-resolution corpus). The
128+
shipped caches stay at 1540. Resolution is a real mechanical lever (RoPE rebuild
129+
works) but is not the end-to-end lever for this corpus -- the remaining failures
130+
are 8B-structuring and GT-annotation-boundary issues, not OCR starvation.

0 commit comments

Comments
 (0)