Skip to content

fix(dspy): support Gemini 3.5 thinking levels, regional prefixes, and streaming usage - #10557

Open
zchryst wants to merge 3 commits into
stanfordnlp:mainfrom
zchryst:fix/gemini-3-5-thinking-and-usage
Open

zchryst wants to merge 3 commits into
stanfordnlp:mainfrom
zchryst:fix/gemini-3-5-thinking-and-usage

Conversation

@zchryst

@zchryst zchryst commented Oct 2, 2026

Copy link
Copy Markdown

Summary

Fixes Gemini 3.x / 3.5 reasoning configuration, regional/resource-path model normalization, and streaming token usage handling in the Gemini provider (dspy/_vendor/lm15/providers/gemini.py):

  1. Regional & resource-path model normalization (gemini_level_class):
    • Strips resource path segments (models/..., publishers/google/models/...) and regional endpoint prefixes (au.gemini-..., eu.gemini-..., us.gemini-...) and matches gemini-3+ models (^gemini-([3-9]|\d{2,})(?:\.\d+|-|$)).
  2. Gemini Pro / Flash thinkingLevel and thinkingBudget constraints (GeminiLM._payload):
    • When reasoning.is_off:
      • On Gemini 3+ Pro models (-pro), substitutes thinkingLevel: "low" (since "minimal" is rejected by Gemini 3 Pro / 3.1 Pro / 3.5 Pro) while keeping "minimal" for Flash models.
      • On Gemini 2.5 Pro, sends thinkingBudget: 128 (minimum supported budget; 0 is rejected) while keeping thinkingBudget: 0 for Flash / Flash-Lite.
    • Clamps effort == "minimal" to "low" on Gemini 3+ Pro models and effort == "medium" to "high" on gemini-3-pro / gemini-3-pro-preview (while preserving "medium" on gemini-3.1-pro and gemini-3.5-pro).
    • Sets thinkingConfig.includeThoughts = True when reasoning is enabled (not was_off and reasoning.summary != "none"), ensuring dspy.Reasoning (adapt_to_native_lm_feature) receives native thought parts from Gemini.
  3. Streaming finish_reason and usage preservation:
    • In GeminiLM.parse_stream_events, trailing usage-only SSE chunks (not yielded_delta and "usageMetadata" in payload) now emit StreamEndEvent(finish_reason=None, ...) so coalesce_stream preserves an earlier finish_reason (such as "tool_call" or "max_tokens") instead of overwriting it with "stop".
    • In GeminiLM._stream_via_live_completion, accumulates reasoning_tokens and cache_read_tokens alongside input_tokens, output_tokens, and total_tokens.

Test Plan

  • Added unit tests in tests/adapters/test_reasoning.py and tests/clients/test_lm15_adaptations_and_timeouts.py.
  • Verified with uv run --extra dev pytest tests/adapters/test_reasoning.py tests/clients/test_lm15_adaptations_and_timeouts.py tests/test_vendored_lm15.py.

@greptile-apps

greptile-apps Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[High risk] Changes how the system recognizes and configures Gemini model versions.

No outstanding findings block merging.

Summary

This PR updates Gemini model normalization, reasoning configuration, and streaming usage handling. The latest change aligns the native-reasoning test with the requirement that visible thoughts be requested explicitly. No new issues were identified.

Reviews (3) · Last reviewed commit: "test(reasoning): align test_reasoning ex..."

Comment thread dspy/_vendor/lm15/providers/gemini.py Outdated
Comment thread dspy/_vendor/lm15/providers/gemini.py

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant