Merge pull request #21 from nshkrdotcom/fix-vertex-models

Fix: update Vertex AI model path handling

Author: nshkrdotcom

CHANGELOG.md

@@ -7,6 +7,85 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.10.0] - 2026-02-22
+
+### Added
+
+#### Model Registry (`Gemini.ModelRegistry`)
+- **Canonical model registry** with capability metadata sourced from Gemini model catalog pages
+- Model lookup by code, resource-style names (`models/`, `publishers/google/models/`), and aliases
+- Capability checks for `:live_api`, `:thinking`, `:function_calling`, `:structured_outputs`, `:audio_generation`, `:image_generation`, `:batch_api`, and `:caching`
+- Support state tracking: `:supported`, `:not_supported`, `:experimental`, `:unknown`
+- Modality tracking (input/output): `:text`, `:image`, `:video`, `:audio`, `:pdf`, `:embeddings`, `:music`
+- Live-model candidate selection by modality (`:text` / `:audio`)
+
+#### Config API Additions
+- **`Gemini.Config.model_info/1`**: Return model metadata from the registry by key atom or model string
+- **`Gemini.Config.model_supports?/2,3`**: Check whether a model supports a given capability
+- **`Gemini.Config.models_with_capability/1,2`**: List model codes matching a capability and support state
+- **`Gemini.Config.live_registry_candidates/1`**: Registry-backed Live API candidate model codes for a modality
+
+#### New Model Entries
+- `gemini-3.1-pro-preview` and `gemini-3.1-pro-preview-customtools` (Gemini 3.1 preview)
+- `gemini-2.5-flash-native-audio-latest` (latest native audio)
+- Legacy Live aliases: `gemini-live-2.5-flash`, `gemini-live-2.5-flash-preview`, `gemini-live-2.5-flash-native-audio`, etc.
+- Specialized model families: `gemini-2.5-computer-use-preview-10-2025`, `gemini-robotics-er-1.5-preview`, `lyria-realtime-exp`, `veo-3.1-generate-preview`, `veo-3.1-fast-generate-preview`, `imagen-4.0-generate-001`, `imagen-4.0-ultra-generate-001`, `imagen-4.0-fast-generate-001`
+
+#### ADC: `GOOGLE_APPLICATION_CREDENTIALS_JSON` Support
+- New ADC credential source for containerized environments (Heroku, Fly.io, Railway, Docker) where mounting credential files is impractical
+- Set `GOOGLE_APPLICATION_CREDENTIALS_JSON='{"type":"service_account",...}'` with full JSON content
+- Parsed before the standard file-path `GOOGLE_APPLICATION_CREDENTIALS` in the credential discovery chain
+- Service account JSON parsing now uses an explicit key allowlist instead of `String.to_atom/1`
+
+#### Vertex AI Model Endpoints
+- **`list_models/1`** uses the correct Vertex publisher models endpoint (`/v1beta1/publishers/google/models`) with `pageSize`/`pageToken` query params
+- **`list_models/1` pagination**: accepts `:page_size` and `:page_token` options passed as query parameters
+- **`get_model/2`** uses `publishers/google/models/{model}` path for Vertex AI
+- **`publisherModels` response normalization**: Vertex AI `list_models` responses are transparently normalized to the same `%ListModelsResponse{}` shape as Gemini API responses
+
+#### HTTP Client Per-Request Auth Override
+- `Gemini.Client.HTTP` verb functions (`get`, `post`, `patch`, `delete`) now respect `:auth`, `:api_key`, `:project_id`, `:location`, `:access_token`, and `:service_account` options passed per-request
+- Absolute paths (`/v1beta1/...`) and full URLs are passed through without re-prefixing
+
+#### Vertex Live Session Model Prefix
+- Live sessions with `auth: :vertex_ai` now build correct `projects/{p}/locations/{l}/publishers/google/models/{m}` resource names in the setup message
+
+#### `stream_generate/2` Configurable Timeout
+- New `:stream_timeout` option (default: 60,000ms, up from hardcoded 30,000ms) for `Gemini.stream_generate/2`
+- Stream cleanup on timeout: `Coordinator.stop_stream/1` is called before returning the error, preventing orphaned streams
+
+#### Live API ADC Integration Test
+- New `test/live_api/adc_live_test.exs` for end-to-end ADC-authenticated Live API sessions
+
+### Changed
+- **Vertex Live API upgraded to v1**: WebSocket endpoint changed from `v1beta1.LlmBidiService/BidiGenerateContent` to `v1.LlmBidiService/BidiGenerateContent`; Vertex WebSocket path no longer appends `?project=...&location=...` query parameters
+- **Live model resolution uses registry**: `Gemini.Live.Models.resolve/1` now consults `ModelRegistry` plus runtime `list_models` results instead of hardcoded candidate lists
+- **Live model auth-aware candidate selection**: separate candidate and default lists for `:gemini` vs `:vertex_ai` auth strategies; Vertex AI candidates exclude Gemini-only `gemini-live-*` aliases; smart modality-based fallback scans available models when no candidate matches
+- **Default text live model**: changed from `gemini-2.0-flash-exp` to `gemini-2.5-flash-native-audio-preview-12-2025`
+- **`latest` model alias**: updated from `gemini-3-pro-preview` to `gemini-3.1-pro-preview`
+- **ADC credential discovery order**: now `GOOGLE_APPLICATION_CREDENTIALS_JSON` → `GOOGLE_APPLICATION_CREDENTIALS` → gcloud user creds → metadata server
+- **`Live.Models.candidates/1`** and **`Live.Models.default/1`** now accept an optional `opts` keyword list for auth-aware candidate selection
+- **`Live.Setup.normalize_model_name/1`**: now recognizes `projects/`, `publishers/`, and `models/` prefixes
+- **Video generation request format**: flattened request params — `numberOfVideos`, `durationSeconds`, `aspectRatio`, `personGeneration` are now top-level keys instead of nested under `videoConfig`; compatible with Veo 3.1 API requirements
+- **Video `person_generation` default**: changed from `:allow_none` to `:dont_allow`; format changed from camelCase (`allowAdult`) to snake_case (`allow_adult`); `:allow_none` now maps to `"dont_allow"`
+- **Video binary encoding**: `image_to_api/1` and `video_to_api/1` now use `%{"inlineData" => %{"data" => ..., "mimeType" => ...}}` format instead of `%{"bytesBase64Encoded" => ...}`; response parsing handles both legacy and new formats
+- **Video generation legacy fields**: `fps`, `compression_format`, `safety_filter_level`, and `guidance_scale` are no longer sent in API requests (struct fields retained for backwards compatibility)
+- **Images API implicit Vertex AI auth**: `generate/3`, `edit/5`, and `upscale/3` now auto-set `auth: :vertex_ai` on opts; validation uses per-request auth credentials via `HTTP.auth_config_for_request/1` with actual credential completeness checking; location resolved from per-request auth config
+- **Streaming options pass-through**: `UnifiedManager` now extracts `:timeout`, `:max_retries`, `:max_backoff_ms`, `:connect_timeout`, `:method`, and `:add_sse_params` from stream config and passes them to `HTTPStreaming.stream_to_process/6`
+- **Model reclassifications**: `deep_research_pro_preview_12_2025` moved from universal to Gemini-only models; `live_2_5_flash_preview` moved to Gemini-only legacy aliases; `flash_2_5_native_audio_latest` added as a universal model
+- **`RUN_BILLED_VERTEX_LIVE_TESTS` gate**: Vertex Live tests now require explicit opt-in via environment variable to prevent accidental GCP billing
+
+### Fixed
+- **Vertex AI model path handling**: `list_models` and `get_model` use correct Vertex AI resource paths instead of Gemini API paths
+- **WebSocket model name normalization**: handles `models/`, `publishers/google/models/`, and full `projects/.../models/` prefixes, plus endpoint suffixes (`:generateContent`)
+- **Live Setup model prefix**: Vertex AI sessions correctly prefix model names with `projects/{p}/locations/{l}/publishers/google/`
+- **HTTP client URL building**: absolute paths and full URLs are no longer double-prefixed with the base URL
+
+### Removed
+- Bulk cleanup of ~50,000 lines of internal planning documents, gap analyses, implementation specs, and development notes not intended for distribution
+- Removed `test.sh`, `repro_concurrency_gate.exs`, and `CLAUDE.md` development artifacts
+- Removed `.kiro/` spec directory
+
 ## [0.9.1] - 2026-01-27
 
 ### Added
@@ -1919,7 +1998,8 @@ config :gemini_ex,
 - Minimal latency overhead
 - Concurrent request processing
 
-[Unreleased]: https://github.com/nshkrdotcom/gemini_ex/compare/v0.9.1...HEAD
+[Unreleased]: https://github.com/nshkrdotcom/gemini_ex/compare/v0.10.0...HEAD
+[0.10.0]: https://github.com/nshkrdotcom/gemini_ex/compare/v0.9.1...v0.10.0
 [0.9.1]: https://github.com/nshkrdotcom/gemini_ex/compare/v0.9.0...v0.9.1
 [0.9.0]: https://github.com/nshkrdotcom/gemini_ex/compare/v0.8.8...v0.9.0
 [0.8.8]: https://github.com/nshkrdotcom/gemini_ex/compare/v0.8.7...v0.8.8

README.md

@@ -18,18 +18,19 @@ A comprehensive Elixir client for Google's Gemini AI API with dual authenticatio
 - **Automatic Tool Calling**: A seamless, Python-SDK-like experience that automates the entire multi-turn tool-calling loop
 - **Built-in Tools (Gemini 3)**: Google Search, URL Context, and Code Execution via `tools:`
 - **Dual Authentication**: Seamless support for both Gemini API keys and Vertex AI OAuth/Service Accounts
-- **Application Default Credentials (ADC)**: Zero-config GCP auth with automatic discovery and token refresh (NEW in v0.8.x!)
+- **Application Default Credentials (ADC)**: Zero-config GCP auth with automatic discovery, token refresh, and `GOOGLE_APPLICATION_CREDENTIALS_JSON` for containers
 - **Advanced Streaming**: Production-grade Server-Sent Events streaming with real-time processing
 - **Interactions API**: Stateful interactions (CRUD), background execution, SSE streaming, and resumption
-- **Live API (WebSocket)**: Bidirectional, low-latency sessions with real-time input/output, native audio with affective dialog and proactivity (Enhanced in v0.9.0!)
+- **Live API (WebSocket)**: Bidirectional, low-latency sessions with real-time input/output, native audio with affective dialog and proactivity (Enhanced in v0.10.0!)
 - **Automatic Rate Limiting**: Built-in rate limit handling with retries, concurrency gating, and adaptive backoff
 - **Files API**: Upload, manage, and use files with Gemini models for multimodal content (NEW in v0.7.0!)
 - **File Search Stores**: RAG store creation, ingestion, and semantic search (NEW in v0.8.x!)
 - **Documents API**: Manage indexed documents inside stores for RAG workflows (NEW in v0.7.0!)
 - **Batches API**: Submit large numbers of requests with 50% cost savings (NEW in v0.7.0!)
 - **Operations API**: Track long-running operations like video generation (NEW in v0.7.0!)
 - **Tunings (Fine-Tuning)**: Create, monitor, and manage tuned models (NEW in v0.8.x!)
-- **Image & Video Generation**: Imagen/Veo APIs for text-to-image, editing, upscaling, and video generation (NEW in v0.8.x!)
+- **Model Registry**: Centralized model capabilities, modality tracking, and registry-backed Live API model resolution (NEW in v0.10.0!)
+- **Image & Video Generation**: Imagen/Veo APIs for text-to-image, editing, upscaling, and video generation with Veo 3.1 support
 - **Embeddings with MRL**: Text embeddings with Matryoshka Representation Learning, normalization, and distance metrics
 - **Async Batch Embeddings**: Production-scale embedding generation with 50% cost savings
 - **Type Safety**: Complete type definitions with runtime validation
@@ -58,7 +59,7 @@ Add `gemini` to your list of dependencies in `mix.exs`:
 ```elixir
 def deps do
   [
-    {:gemini_ex, "~> 0.9.1"}
+    {:gemini_ex, "~> 0.10.0"}
   ]
 end
 ```
@@ -175,7 +176,7 @@ Gemini.Streaming.resume_stream(stream_id)
 Gemini.Streaming.stop_stream(stream_id)
 ```
 
-Streaming knobs: pass `timeout:` (per attempt, default `config :gemini_ex, :timeout` = 120_000), `max_retries:` (default 3), `max_backoff_ms:` (default 10_000), and `connect_timeout:` (default 5_000). Manager cleanup delay can be tuned via `config :gemini_ex, :streaming, cleanup_delay_ms: ...`.
+Streaming knobs: pass `timeout:` (per attempt, default `config :gemini_ex, :timeout` = 120_000), `stream_timeout:` (collect timeout, default 60_000; orphaned streams are cleaned up on timeout), `max_retries:` (default 3), `max_backoff_ms:` (default 10_000), and `connect_timeout:` (default 5_000). Manager cleanup delay can be tuned via `config :gemini_ex, :streaming, cleanup_delay_ms: ...`.
 
 ### Interactions Quick Start
 
@@ -201,15 +202,15 @@ for event <- stream do
 end
 ```
 
-See `docs/guides/interactions.md` for CRUD, resumption (`last_event_id`), and background/cancel/delete examples.
+See `guides/interactions.md` for CRUD, resumption (`last_event_id`), and background/cancel/delete examples.
 
 ### Live API (WebSocket)
 
-Real-time bidirectional streaming for voice, video, and text interactions. The v0.9.0 release upgrades to v1beta as the default API version while supporting v1alpha for advanced native audio features.
+Real-time bidirectional streaming for voice, video, and text interactions. For Gemini Live connections, `v1beta` is the default API version, while `v1alpha` is available for advanced native-audio features. Vertex Live connections use the Vertex `v1` WebSocket endpoint.
 
 #### Model Resolution
 
-Live API model availability varies by API key and regional rollout. Use `Gemini.Live.Models.resolve/1` to automatically select an available model:
+Live API model availability varies by API key and regional rollout. `Gemini.Live.Models.resolve/1` uses the model registry plus runtime `list_models` results to select a compatible model:
 
 ```elixir
 alias Gemini.Live.Models
@@ -284,7 +285,7 @@ tools = [
 )
 ```
 
-See the [Live API Guide](docs/guides/live_api.md) for complete documentation including voice activity detection, session resumption, thinking budgets, and context window compression.
+See the [Live API Guide](guides/live_api.md) for complete documentation including voice activity detection, session resumption, thinking budgets, and context window compression.
 
 ### Rate Limiting & Concurrency (built-in)
 
@@ -393,7 +394,7 @@ config =
   |> GenerationConfig.property_ordering(["answer", "confidence"])
 ```
 
-See [Structured Outputs Guide](docs/guides/structured_outputs.md) for details.
+See [Structured Outputs Guide](guides/structured_outputs.md) for details.
 
 ## Context Caching (New in v0.6.0!)
 
@@ -468,7 +469,7 @@ alias Gemini.Types.File
 - Automatic MIME type detection
 - 48-hour file expiration
 
-See [Files API Guide](docs/guides/files.md) for complete documentation.
+See [Files API Guide](guides/files.md) for complete documentation.
 
 ## File Search Stores (New in v0.8.x!)
 
@@ -559,7 +560,7 @@ end
 - GCS and BigQuery integration (Vertex AI)
 - Comprehensive job management
 
-See [Batches API Guide](docs/guides/batches.md) for complete documentation.
+See [Batches API Guide](guides/batches.md) for complete documentation.
 
 ## Operations API (New in v0.7.0!)
 
@@ -596,7 +597,7 @@ end
 - Cancel and delete operations
 - Comprehensive state helpers
 
-See [Operations API Guide](docs/guides/operations.md) for complete documentation.
+See [Operations API Guide](guides/operations.md) for complete documentation.
 
 ## Tunings API (New in v0.8.x!)
 
@@ -1360,6 +1361,8 @@ export GEMINI_API_KEY="your_gemini_api_key"
 
 # For Vertex AI (optional, for multi-auth demos)
 export VERTEX_JSON_FILE="/path/to/service-account.json"
+# Alternative (standard ADC path)
+export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account.json"
 export VERTEX_PROJECT_ID="your-gcp-project-id"
 ```
 
@@ -1408,16 +1411,21 @@ config :gemini_ex, :auth,
 
 Zero-config GCP authentication with automatic credential discovery and token refresh.
 
+```bash
+# Configure ADC explicitly (optional)
+export GOOGLE_APPLICATION_CREDENTIALS_JSON='{"type":"service_account",...}'  # gemini_ex extension
+export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service_account.json"
+```
+
 ```elixir
 # Works on GCE/Cloud Run/GKE with no extra setup
 {:ok, response} = Gemini.generate("Hello from Vertex AI", auth: :vertex_ai)
 
-# Or point to a service account key
-export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service_account.json"
+# Also works with either env var above
 {:ok, response} = Gemini.generate("Hello", auth: :vertex_ai)
 ```
 
-The client checks `GOOGLE_APPLICATION_CREDENTIALS`, gcloud user credentials, and metadata server endpoints, caching access tokens for you via ETS.
+The client checks `GOOGLE_APPLICATION_CREDENTIALS_JSON` (gemini_ex extension), `GOOGLE_APPLICATION_CREDENTIALS` (standard ADC), gcloud user credentials, and metadata server endpoints, caching access tokens for you via ETS. Official Google ADC order starts with `GOOGLE_APPLICATION_CREDENTIALS`; JSON-content env support is a gemini_ex convenience.
 
 ## Model Configuration System
 
@@ -1459,6 +1467,11 @@ Gemini.Config.models_for(:both)       # Only universal models
 # Get model by key with validation
 Gemini.Config.get_model(:flash_2_5)  #=> "gemini-2.5-flash"
 Gemini.Config.get_model(:flash_2_5, api: :vertex_ai)  # Validates compatibility
+
+# Registry metadata helpers
+Gemini.Config.model_info(:pro_3_1_preview)
+Gemini.Config.model_supports?(:pro_3_1_preview, :thinking)     #=> true
+Gemini.Config.models_with_capability(:live_api, :supported)
 ```
 
 ### Embedding Model Differences
@@ -1507,7 +1520,7 @@ Gemini.embed_content("Text", model: "gemini-embedding-001")
 - **[Architecture Guide](https://hexdocs.pm/gemini_ex/architecture.html)** - System design and components
 - **[Authentication System](https://hexdocs.pm/gemini_ex/authentication_system.html)** - Detailed auth configuration
 - **[Examples](https://github.com/nshkrdotcom/gemini_ex/tree/main/examples)** - Working code examples
-- **Guides (docs/guides/...)**:
+- **Guides (guides/...)**:
   - `adc.md` - Application Default Credentials
   - `batches.md` - Batches API
   - `file_search_stores.md` - RAG stores and document ingestion
@@ -1652,9 +1665,9 @@ content = [
 
 **Supported image formats:** PNG, JPEG, GIF, WebP (auto-detected from magic bytes)
 
-### Image Generation API (Imagen, New in v0.8.x!)
+### Image Generation API (Imagen)
 
-Use the dedicated Imagen endpoints for text-to-image, editing, and upscaling (Vertex AI).
+Use the dedicated Imagen endpoints for text-to-image, editing, and upscaling. As of v0.10.0, `auth: :vertex_ai` is set automatically on all Images API calls.
 
 ```elixir
 alias Gemini.APIs.Images
@@ -1704,7 +1717,7 @@ alias Gemini.Types.Generation.Video.VideoGenerationConfig
 {:ok, op} =
   Videos.generate(
     "A cinematic drone shot over misty mountains at sunrise",
-    %VideoGenerationConfig{duration_seconds: 6, aspect_ratio: "16:9"},
+    %VideoGenerationConfig{duration_seconds: 8, aspect_ratio: "16:9"},
     auth: :vertex_ai
   )