Keep the last-resort fallback pass penalty-free

The repetition penalty is a lossy transform justified only where it turns
a failure into a success -- which it does on the retry (16083265: [ERR] ->
16/16 sections). On the fallback it rescues no poster and measurably
degrades already-failing output: the RTL poster 8228476 dropped from
rougeL 0.71 to 0.57 with the fallback penalty. Applying it on the retry
only fixes 16083265 while restoring 8228476 to its penalty-free baseline
(0.71). Doc updated.

Author: jimnoneill

llama_generation_settings.md

@@ -67,13 +67,13 @@ The model's native context is 128K, so 15k-in + 24k-out sits comfortably inside
 
 1. **Primary** — full `EXTRACTION_PROMPT` @ `MAX_JSON_TOKENS` (18k), plain greedy (no penalty).
 2. **Retry** — same full prompt @ `MAX_RETRY_TOKENS` (24k) with `repetition_penalty = RETRY_REPETITION_PENALTY`, if step 1 errored, was truncated, or hit the cap without EOS.
-3. **Fallback** — shorter `FALLBACK_PROMPT` @ `MAX_RETRY_TOKENS` (24k), same penalty, if step 2 still failed those checks.
+3. **Fallback** — shorter `FALLBACK_PROMPT` @ `MAX_RETRY_TOKENS` (24k), **plain greedy (no penalty)**, if step 2 still failed those checks.
 
 Each step is followed by `_robust_json_parse()` (hand-rolled repair passes, then the `json-repair` library as a last resort). The ladder is cheap in the common case — most posters succeed on step 1 and never pay for the retries.
 
 **The `hit_eos` trigger.** `_generate()` returns `(text, hit_eos)`; `hit_eos` is False when generation reached `max_new_tokens` without the brace processor ever letting an EOS through — the signature of a runaway that never closed the JSON. This condition is required in addition to the parse-error check because `_robust_json_parse()` is strong enough to salvage a truncated runaway into a *parseable but gutted* object (e.g. a single section), which carries no `"error"` key and would otherwise skip the retry that actually repairs it.
 
-**Why a repetition penalty only on retry.** A few posters drive the stock 8B into a repetition loop — it re-emits the `creators` array indefinitely and never closes the top-level object, grinding to the 18k cap and yielding `[ERR]`. A gentle `repetition_penalty` breaks the loop. It is applied **only on the retry/fallback passes**: healthy posters emit EOS on the primary pass and never reach the retry, so their output stays byte-identical to plain greedy. The value **1.15** was tuned empirically — 1.3 overshoots, corrupting the verbatim transcript the task depends on (`Genta -> Gentaa`, `drug polymer interactions -> drugpolymerinteractions`), while 1.15 breaks the loop and recovers a clean, fully-sectioned extraction. On the poster that motivated this (16083265), it moved the end-to-end result from `[ERR]` (0 sections) to a passing extraction (16/16 sections, rougeL 0.86).
+**Why a repetition penalty only on retry.** A few posters drive the stock 8B into a repetition loop — it re-emits the `creators` array indefinitely and never closes the top-level object, grinding to the 18k cap and yielding `[ERR]`. A gentle `repetition_penalty` breaks the loop. It is applied **only on the retry pass** (step 2), not the primary or the fallback: healthy posters emit EOS on the primary pass and never reach the retry, so their output stays byte-identical to plain greedy; and the penalty is a lossy transform justified only where it converts a failure into a success, which it does on the retry but not on the last-resort fallback (there it only degrades already-failing output, e.g. the RTL poster 8228476, so the fallback stays penalty-free). The value **1.15** was tuned empirically — 1.3 overshoots, corrupting the verbatim transcript the task depends on (`Genta -> Gentaa`, `drug polymer interactions -> drugpolymerinteractions`), while 1.15 breaks the loop and recovers a clean, fully-sectioned extraction. On the poster that motivated this (16083265), it moved the end-to-end result from `[ERR]` (0 sections) to a passing extraction (16/16 sections, rougeL 0.86).
 
 ## Quantization
 

poster2json/extract.py

@@ -3190,14 +3190,16 @@ def extract_json_with_retry(
         )
         result = _as_result_dict(_robust_json_parse(response))
 
-    # Fallback to shorter prompt
+    # Fallback to a shorter prompt. This last-resort pass stays penalty-free
+    # (plain greedy): the repetition penalty is a lossy transform justified only
+    # where it converts a failure into a success, which it does on the retry
+    # above. On the fallback no poster is rescued by it, and it measurably
+    # degrades already-failing output (e.g. the RTL poster 8228476), so it is
+    # not applied here.
     if "error" in result or not hit_eos or _is_truncated(result.get("raw", "")):
         log("Using fallback shorter prompt")
         fallback_prompt = FALLBACK_PROMPT.format(raw_text=raw_text)
-        response, hit_eos = _generate(
-            model, tokenizer, fallback_prompt, MAX_RETRY_TOKENS,
-            repetition_penalty=RETRY_REPETITION_PENALTY,
-        )
+        response, hit_eos = _generate(model, tokenizer, fallback_prompt, MAX_RETRY_TOKENS)
         result = _as_result_dict(_robust_json_parse(response))
 
     result = _postprocess_json(