feat(gemini): adopt 3.7 Flash and clamp thinking levels per model

Gemini 3.7 Flash replaces 3.6 everywhere a Flash run can be started: the
four text tiers, GEMINI_DOCUMENT_MODELS, HTR, magazine-issue extraction,
YouTube transcription's non-French slot, and the audio/video pipelines.
Its authority item is 111774, verified live along with the annotation
payloads for iwac:ocrModel, iwac:summaryModel and iwac:transcriptionModel.

3.7 dropped the MINIMAL thinking level, and gemini-flash-latest rolled
onto 3.7 the same day. Every Gemini path here hardcoded MINIMAL, so OCR
extraction, HTR, audio, video and all four text tiers were returning 400
INVALID_ARGUMENT on every call before this change. ModelOption now carries
supported_thinking_levels, filled in from a live probe of all nine
Gemini/Gemma models rather than inferred from their names, and
clamp_thinking_level() snaps a request to the nearest rung the model has,
rounding up on a tie. Pipelines go on asking for "minimal" to mean "as
little as this model offers". The clamp replaces the hand-coded Gemma
MINIMAL/HIGH mapping in the adapter and is shared with gemini_utils, which
is what fixes the multimodal side.

Tiers now offer the pinned gemini-3.7-flash rather than the rolling
gemini-flash: whatever a tier picks is what step 03 stamps into an
iwac:*Model annotation, and a rolling alias reports its own version as the
string "Gemini Flash Latest". The rolling entry stays in MODEL_REGISTRY for
runs that stamp nothing, and TEXT_FULL_MODELS keeps it on purpose because
OCR correction writes no annotation of its own.

A new test asserts every Gemini key a provenance-stamping tier can pick
also has an authority item. It failed on the rolling gemini-pro, which sat
in GEMINI_DOCUMENT_MODELS even though AI_ocr_extraction/03 stamps
iwac:ocrModel — same bug class, so that is pinned to gemini-3.1-pro too.

gemini-3.6-flash stays in AI_MODEL_ITEMS: 10 YouTube transcripts already on
the archive annotate item 79611, and OCR text produced by 3.6 that has not
reached step 03 must be stamped for the model that actually read it. Three
display_title values that had drifted from their live Omeka titles are
corrected alongside it (3.1 pro, 3.6 flash, 3.1 flash lite).

The Mooré comparison and throughput figures in the YouTube README were
measured on 3.6 and are left attributed to it rather than re-attributed to
a model that has not been benchmarked here.

Author: fmadore

AI_NER/01_NER_AI.py

@@ -3,7 +3,7 @@
 
 Supported models:
   - gpt-5.6-luna: Fast, cost-effective OpenAI model (GPT-5.6 high-volume tier)
-  - gemini-flash: Fast, cost-effective Gemini model
+  - gemini-3.7-flash: Fast, cost-effective Gemini model
   - gemma-4: Google Gemma 4 31B — open-weights flagship, via Gemini API
   - mistral-large: Mistral Large 3 flagship model
   - ministral-14b: Ministral 3 14B cost-effective model
@@ -29,7 +29,7 @@
 Usage examples:
     python 01_NER_AI.py --item-set-id 123
     python 01_NER_AI.py --item-set-id 123 --model gpt-5.6-luna
-    python 01_NER_AI.py --item-set-id 123 --model gemini-flash --async
+    python 01_NER_AI.py --item-set-id 123 --model gemini-3.7-flash --async
     python 01_NER_AI.py --item-set-id 123 --model gemma-4
     python 01_NER_AI.py --item-set-id 123 --model mistral-large
     python 01_NER_AI.py --item-set-id 123 --model ministral-14b

AI_NER/README.md

@@ -50,7 +50,7 @@ different provenance; use `--force` to replace it deliberately.
 
 | Model | Provider | Speed | Cost |
 |-------|----------|-------|------|
-| `gemini-flash` | Google | Fast | Low |
+| `gemini-3.7-flash` | Google | Fast | Low |
 | `gemma-4` | Google (Gemma 4 31B, open-weights via Gemini API) | Fast | Low |
 | `gpt-5.6-luna` | OpenAI | Fast | Low |
 | `mistral-large` | Mistral | Medium | Medium |
@@ -61,7 +61,7 @@ different provenance; use `--force` to replace it deliberately.
 
 All models use the same French-language prompt (`ner_system_prompt.md`) optimized for West African Islamic contexts.
 
-`gemma-4` uses the same `GEMINI_API_KEY` as the Gemini models. Thinking level is `minimal` by default (Gemma 4 accepts only `MINIMAL` or `HIGH`), which matches the low-cost entity-extraction budget used by `gemini-flash`.
+`gemma-4` uses the same `GEMINI_API_KEY` as the Gemini models. Thinking level is `minimal` by default (Gemma 4 accepts only `MINIMAL` or `HIGH`), which matches the low-cost entity-extraction budget used by `gemini-3.7-flash`.
 
 `qwen3.5-moe` and `deepseek-v4-flash-0731` share one `OPENROUTER_API_KEY` and cost
 roughly a tenth of `gpt-5.6-luna`, which is what makes a full-corpus pass

AI_audio_summary/02_AI_transcribe_audio.py

@@ -104,8 +104,14 @@ def __init__(
 
         Args:
             api_key (str, optional): Gemini API key. If None, will use GEMINI_API_KEY environment variable.
-            model (str, optional): Model to use — 'gemini-pro-latest', 'gemini-flash-latest', or
+            model (str, optional): Model to use — 'gemini-pro-latest', 'gemini-3.7-flash', or
                 'gemini-flash-lite-latest'. Default is 'gemini-pro-latest'.
+                The Flash slot names a pinned release where the other two roll,
+                because each transcript records ``Generated using: Google
+                <model>`` in its header — and "gemini-flash-latest" names no
+                version a reader could look up later. Pro and Flash-Lite are
+                still rolling for now; pin them when their headers start being
+                cited too.
             requests_per_minute: Optional RPM limit for proactive throttling (None = no throttling)
             transcription_prompt: The transcription prompt to use (selected in ``main()``);
                 falls back to ``DEFAULT_PROMPT``.
@@ -712,7 +718,7 @@ def parse_args():
     )
     parser.add_argument(
         "--model",
-        choices=["gemini-pro-latest", "gemini-flash-latest", "gemini-flash-lite-latest"],
+        choices=["gemini-pro-latest", "gemini-3.7-flash", "gemini-flash-lite-latest"],
         default=None,
         help="Model to use for transcription (default: interactive selection)"
     )
@@ -754,7 +760,7 @@ def select_model_interactive():
     models_table.add_column("Model", style="green")
     models_table.add_column("Description", style="dim")
     models_table.add_row("1", "gemini-pro-latest", "Higher quality, slower")
-    models_table.add_row("2", "gemini-flash-latest", "Faster, good quality")
+    models_table.add_row("2", "gemini-3.7-flash", "Faster, good quality")
     models_table.add_row("3", "gemini-flash-lite-latest", "Fastest, cheapest, lowest latency")
     console.print(models_table)
 
@@ -763,7 +769,7 @@ def select_model_interactive():
     ).strip()
 
     if model_choice == '2':
-        return 'gemini-flash-latest'
+        return 'gemini-3.7-flash'
     if model_choice == '3':
         return 'gemini-flash-lite-latest'
     return 'gemini-pro-latest'

AI_audio_summary/README.md

@@ -69,7 +69,7 @@ Select mode interactively or edit prompt files in `prompts/`.
 | Model | Speed | Quality | Cost |
 |-------|-------|---------|------|
 | `gemini-flash-lite-latest` | Fastest | Good | ~$0.20-1/hour |
-| `gemini-flash-latest` | Faster | Good | ~$0.50-2/hour |
+| `gemini-3.7-flash` | Faster | Good | ~$0.50-2/hour |
 | `gemini-pro-latest` | Slower | Higher | ~$5-15/hour |
 
 Use Flash-Lite for clean, single-speaker recordings on a budget, Flash for general use, Pro for noisy audio or multiple speakers.

AI_htr_extraction/README.md

@@ -49,7 +49,7 @@ Each mode uses a specialized prompt (`htr_system_prompt_*.md`) with language-spe
 
 | Model | Speed | Accuracy | Best For |
 |-------|-------|----------|----------|
-| Gemini Flash | Faster | Good | Clear handwriting, bulk processing |
+| Gemini 3.7 Flash | Faster | Good | Clear handwriting, bulk processing |
 | Gemini Pro | Slower | Higher | Difficult scripts, faded ink |
 
 ## Output

AI_htr_extraction/gemini_htr_processor.py

@@ -72,8 +72,11 @@
     "multilingual": ("text (detect language automatically)", "htr_system_prompt_multilingual.md"),
 }
 
-# Gemini models available for HTR, keyed by llm_provider registry key.
-HTR_MODELS = ["gemini-flash", "gemini-pro"]
+# Gemini models available for HTR, keyed by llm_provider registry key. Flash is
+# the pinned 3.7 rather than the rolling ``gemini-flash``: HTR output is written
+# back under an ``iwac:ocrModel`` annotation, and a rolling alias reports its own
+# version as "Gemini Flash Latest".
+HTR_MODELS = ["gemini-3.7-flash", "gemini-3.1-pro"]
 
 
 def load_system_instruction(language: str) -> str:

AI_ocr_correction/02_correct_alto_xml.py

@@ -14,7 +14,7 @@
 
 Supports multiple models via --model flag:
 - deepseek-v4-flash-0731 (default): DeepSeek official Flash release via OpenRouter
-- gemini-flash: Gemini Flash - fast, cost-effective
+- gemini-3.7-flash: Gemini 3.7 Flash - fast, cost-effective
 - gemini-pro: Gemini Pro - highest quality
 - gpt-5.6-luna: OpenAI GPT-5.6 Luna - cost-optimized tier
 - gpt-5.6-sol: OpenAI GPT-5.6 Sol - flagship tier
@@ -625,7 +625,7 @@ def parse_args() -> argparse.Namespace:
   python 02_correct_alto_xml.py
 
   # Use Gemini Flash (fast, no thinking)
-  python 02_correct_alto_xml.py --model gemini-flash
+  python 02_correct_alto_xml.py --model gemini-3.7-flash
 
   # Custom directories and max lines per request
   python 02_correct_alto_xml.py --input-dir ./ALTO --output-dir ./ALTO_Corrected --max-lines 40
@@ -690,7 +690,9 @@ def main():
     # Configure LLM based on provider. Temperature is deliberately absent: it is
     # set per model in MODEL_REGISTRY from each vendor's guidance, and forcing it
     # low here is what Gemini 3 and Qwen both document as a cause of looping.
-    if model_option.key == "gemini-flash":
+    # "minimal" here means "as little as this model offers" — Gemini 3.7 Flash
+    # dropped that rung, and the registry clamp turns the request into LOW.
+    if model_option.key == "gemini-3.7-flash":
         config = LLMConfig(thinking_level="minimal")  # Fastest/cheapest
     elif model_option.key == "gemini-pro":
         config = LLMConfig(thinking_level="low")  # Minimal thinking
@@ -707,8 +709,8 @@ def main():
     config_table.add_row("Input Directory", str(input_dir))
     config_table.add_row("Output Directory", str(output_dir))
     config_table.add_row("Max Lines/Request", f"{args.max_lines} (for large blocks)")
-    if model_option.key == "gemini-flash":
-        config_table.add_row("Thinking", "Minimal (thinking_level=minimal)")
+    if model_option.key == "gemini-3.7-flash":
+        config_table.add_row("Thinking", "Shallowest available (thinking_level=low)")
     elif model_option.key == "gemini-pro":
         config_table.add_row("Thinking Level", "low")
     console.print(config_table)

AI_ocr_correction/02_correct_ocr_text.py

@@ -7,7 +7,7 @@
 
 Supports multiple models via --model flag:
 - deepseek-v4-flash-0731 (default): DeepSeek official Flash release via OpenRouter
-- gemini-flash: Gemini Flash - fast, cost-effective
+- gemini-3.7-flash: Gemini 3.7 Flash - fast, cost-effective
 - gemini-pro: Gemini Pro - highest quality
 - gpt-5.6-luna: OpenAI GPT-5.6 Luna - cost-optimized tier
 - gpt-5.6-sol: OpenAI GPT-5.6 Sol - flagship tier
@@ -308,11 +308,12 @@ def main():
         sys.exit(1)
 
     # Configure LLM based on provider
-    # For Gemini Flash: disable thinking for faster, cheaper processing
+    # For Gemini Flash: as little thinking as the model offers, for speed/cost.
+    # 3.7 dropped MINIMAL, so the registry clamp turns that request into LOW.
     # For other models: use appropriate defaults
     # Temperature is left to MODEL_REGISTRY, which carries each vendor's own
     # recommendation — pinning it low here is a documented cause of looping.
-    if model_option.key == "gemini-flash":
+    if model_option.key == "gemini-3.7-flash":
         config = LLMConfig(thinking_level="minimal")  # Fastest/cheapest
     elif model_option.key == "gemini-pro":
         config = LLMConfig(thinking_level="low")  # Minimal thinking
@@ -329,8 +330,8 @@ def main():
     config_table.add_row("Input Directory", str(input_dir))
     config_table.add_row("Output Directory", str(output_dir))
     config_table.add_row("Max Chunk Length", f"{args.max_length:,} chars")
-    if model_option.key == "gemini-flash":
-        config_table.add_row("Thinking", "Minimal (thinking_level=minimal)")
+    if model_option.key == "gemini-3.7-flash":
+        config_table.add_row("Thinking", "Shallowest available (thinking_level=low)")
     elif model_option.key == "gemini-pro":
         config_table.add_row("Thinking Level", "low")
     console.print(config_table)

AI_ocr_correction/README.md

@@ -42,15 +42,15 @@ provenance survives.
 
 ```bash
 # Place ALTO XML files in ALTO/ folder
-python 02_correct_alto_xml.py --model gemini-flash
+python 02_correct_alto_xml.py --model gemini-3.7-flash
 # Corrected files saved to ALTO_Corrected/
 ```
 
 ## Supported Models
 
 | Model | Speed | Best For |
 |-------|-------|----------|
-| `gemini-flash` | Fast | Bulk processing |
+| `gemini-3.7-flash` | Fast | Bulk processing |
 | `gemini-pro` | Slower | Complex documents |
 | `gpt-5.6-luna` | Fast | General use |
 | `gpt-5.6-sol` | Slower | Higher quality |

AI_ocr_extraction/02_gemini_ocr_processor.py

@@ -21,7 +21,8 @@
 
 Model Selection:
     - Uses shared LLM provider with three options (all via the Gemini API):
-        * Gemini Flash — faster, cost-effective, uses MINIMAL thinking
+        * Gemini 3.7 Flash — faster, cost-effective, uses LOW thinking
+          (3.7 dropped MINIMAL; LOW is the shallowest rung it has.)
         * Gemini Pro — higher quality, uses LOW thinking
         * Gemma 4 31B    — open-weights flagship, uses MINIMAL thinking
           (Gemma 4 accepts only MINIMAL or HIGH; MINIMAL is used for OCR speed.)
@@ -133,14 +134,15 @@ def main():
 
     # OCR runs at the model's default temperature: Google recommends sending none
     # for Gemini 3, because a lowered one can make the model loop — on a page scan
-    # that shows up as the same line repeating until max_output_tokens. Minimal
-    # thinking for speed. All Gemini 3 / Gemma 4 models use thinking_level (it
-    # cannot be disabled):
-    #   Gemini Flash: MINIMAL — fastest, sufficient for OCR
-    #   Gemini Pro:   LOW     — Pro does not accept MINIMAL
-    #   Gemma 4:      MINIMAL — only MINIMAL or HIGH accepted; MINIMAL for speed
+    # that shows up as the same line repeating until max_output_tokens.
+    #
+    # Ask for the shallowest thinking there is and let the registry's clamp find
+    # it: thinking cannot be disabled on any of these models, and which rungs
+    # exist is per-model and changes between releases (Gemini 3.7 Flash dropped
+    # MINIMAL; Gemma 4 has only MINIMAL and HIGH). Naming the rung here is how
+    # this line used to encode "Flash means MINIMAL", which stopped being true.
     llm_config = LLMConfig(
-        thinking_level="LOW" if "pro" in model_option.model.lower() else "MINIMAL",
+        thinking_level=get_thinking_level(model_option.model, "MINIMAL"),
     )
 
     console.print(key_value_table([