Skip to content

Watch Gemini API stability (v1beta path, usageMetadata field names, thinking-token semantics) #68

Description

@sylvester-francis

leash's native Gemini support depends on a few things Google has a habit of changing. Tracking them here so a silent metering regression gets caught, not shipped.

What leash depends on (and how fragile each is):

  • The path. Detection keys on a path containing generatecontent (case-insensitive), so a version-prefix bump (/v1beta/ -> /v1/) does not break detection. Robust. But the examples and docs reference /v1beta/ and generativelanguage.googleapis.com explicitly; update those if the version moves.
  • The usageMetadata field names (promptTokenCount, candidatesTokenCount, thoughtsTokenCount, cachedContentTokenCount, totalTokenCount) in internal/meter/parse.go. Fragile: if Google renames or restructures these, the parser reads zeros, so a real call goes blind (fail-closed under a budget) or undercounts. No error is raised.
  • The thinking-token semantics. On the Gemini API candidatesTokenCount includes thoughtsTokenCount (we map thoughts -> reasoning, priced once); on Vertex AI it excludes them (tracked separately in Support Vertex AI Gemini usageMetadata semantics #62). If Google changes the Gemini-API behavior to match Vertex, leash would undercount output by the thinking amount.

Suggested guard: a periodic check (manual, or a tiny scheduled job) that hits a real Gemini generateContent response fixture and asserts the field names and the candidates-includes-thoughts relationship still hold. Until then, the offline examples/demos/11-gemini.sh and the meter tests pin the shapes we currently expect, so a mismatch surfaces the moment we update the fixture.

Related: #62 (Vertex AI semantics).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    good first issueGood for newcomershelp wantedExtra attention is neededroadmapDeferred capability tracked in docs/known-issues.md

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions