feat: Add Oracle Cloud Infrastructure (OCI) Generative AI client support by fede-kamel · Pull Request #718 · cohere-ai/cohere-python

fede-kamel · 2026-01-26T03:37:16Z

Overview

I noticed that the Cohere Python SDK has excellent integration with AWS Bedrock through the BedrockClient implementation. I wanted to contribute a similar integration for Oracle Cloud Infrastructure (OCI) Generative AI service to provide our customers with the same seamless experience.

Motivation

Oracle Cloud Infrastructure offers Cohere's models through our Generative AI service, and many of our enterprise customers use both platforms. This integration follows the same architectural pattern as the existing Bedrock client, ensuring consistency and maintainability.

Implementation

This PR adds comprehensive OCI support with:

Features

OciClient (V1 API) and OciClientV2 (V2 API) classes
Full authentication support:
- Config file (default ~/.oci/config)
- Custom profiles
- Direct credentials
- Instance principal (for OCI compute instances)
- Resource principal
Complete API coverage:
- Embed (all models: english-v3.0, light-v3.0, multilingual-v3.0)
- Chat with streaming support (Command R and Command A models)
- V2 API support with Command A models (command-a-03-2025)
Region-independent: Uses display names instead of region-specific OCIDs
Automatic V1/V2 API detection and transformation

Architecture

Follows the proven BedrockClient pattern with httpx event hooks
Request/response transformation between Cohere and OCI formats
Lazy loading of OCI SDK as optional dependency
Connection pooling for optimal performance

Testing

14 comprehensive integration tests (100% passing)
Tests cover: authentication, embed, chat, chat_stream, error handling
Multiple model variants tested

Documentation

README section with usage examples
All authentication methods documented
Installation instructions for optional OCI dependency

Files Changed

src/cohere/oci_client.py (910 lines) - Main OCI client implementation
src/cohere/manually_maintained/lazy_oci_deps.py (30 lines) - Lazy OCI SDK loading
tests/test_oci_client.py (393 lines) - Comprehensive integration tests
README.md - OCI usage documentation
pyproject.toml - Optional OCI dependency
src/cohere/__init__.py - Export OciClient and OciClientV2

Test Results

14 passed, 8 skipped, 0 failed

Skipped tests are for OCI service limitations (base models not callable via on-demand inference).

Breaking Changes

None. This is a purely additive feature.

Checklist

Code follows repository style (ruff passing)
Tests added and passing
Documentation updated
No breaking changes

Note

Medium Risk
Adds a large new OCI integration layer that rewrites and signs outbound requests and transforms streaming/non-streaming responses, which could introduce subtle compatibility or parsing bugs despite being additive.

Overview
Adds first-class Oracle Cloud Infrastructure support via new OciClient (v1) and OciClientV2 (v2) that route Cohere embed/chat (+ chat_stream) calls through OCI’s Generative AI inference endpoints using httpx event hooks.

Implements OCI auth and request signing (config-file profiles, direct credentials, instance/resource principals, and session-token refresh), plus bidirectional payload/stream event translation between Cohere and OCI formats (including model-name normalization and v2 content-block lifecycle handling).

Exposes the new clients from cohere.__init__, adds optional oci dependency/extras in pyproject.toml, documents usage in README.md, and introduces a comprehensive tests/test_oci_client.py suite covering auth, transformations, and live integration (gated by env vars).

^{Reviewed by Cursor Bugbot for commit a06825a. Bugbot is set up for automated code reviews on this repo. Configure here.}

fede-kamel · 2026-01-26T16:30:07Z

@walterbm-cohere @daniel-cohere @billytrend-cohere

Hey maintainers,

Friendly bump on this PR - would appreciate your feedback when you have a chance. Happy to address any concerns or make changes as needed.

Thanks.

fede-kamel · 2026-02-24T23:10:48Z

Addressed Bugbot feedback:

V2 streaming ends with wrong event (High) - Now emits message-end event before returning on [DONE]
Direct OCI credentials can crash (Medium) - Added validation: when oci_user_id is provided, oci_fingerprint and oci_tenancy_id are now required with a clear error message

fede-kamel · 2026-02-26T17:55:59Z

@sanderland Thanks for the approvals on this PR and the others (#717, #698, #697)! What are the next steps to get these merged?

fede-kamel · 2026-03-03T17:05:15Z

@sanderland quick ping on this one since you approved earlier - could you please take a final look when you have a moment? Thanks!

fede-kamel · 2026-03-15T18:53:51Z

@billytrend-cohere @walterbm-cohere @daniel-cohere @sanderland

This PR is the canonical Oracle proposal for OCI support in cohere-python, and it now reflects the review feedback that was actionable for the OCI integration scope.

What is in scope on this branch:

OciClient and OciClientV2 support for OCI authentication flows
embed, chat, and streaming support
V2 request/response and streaming handling aligned with the existing SDK surface
focused OCI regression coverage and live OCI validation

Current verification on this branch:

focused OCI/AWS suite passes locally
live OCI validation passes for the runnable cases under the available OCI profiles
remaining skips are expected OCI model or region availability gaps, not code failures

Please treat #718 as the official Oracle OCI integration proposal going forward. If there are any remaining blockers to merge, please call them out directly on this PR and I will address them here.

fede-kamel · 2026-03-24T12:52:49Z

Final test run — 47/47 passed

All tests passing against the live OCI GenAI inference layer (us-chicago-1):

tests/test_oci_client.py::TestOciClient::test_chat PASSED                [  2%]
tests/test_oci_client.py::TestOciClient::test_chat_stream PASSED         [  4%]
tests/test_oci_client.py::TestOciClient::test_embed PASSED               [  6%]
tests/test_oci_client.py::TestOciClientV2::test_chat_stream_v2 PASSED    [  8%]
tests/test_oci_client.py::TestOciClientV2::test_chat_v2 PASSED           [ 10%]
tests/test_oci_client.py::TestOciClientV2::test_embed_v2 PASSED          [ 12%]
tests/test_oci_client.py::TestOciClientV2::test_embed_with_model_prefix_v2 PASSED [ 14%]
tests/test_oci_client.py::TestOciClientAuthentication::test_config_file_auth PASSED [ 17%]
tests/test_oci_client.py::TestOciClientAuthentication::test_custom_profile_auth PASSED [ 19%]
tests/test_oci_client.py::TestOciClientErrors::test_invalid_model PASSED [ 21%]
tests/test_oci_client.py::TestOciClientErrors::test_missing_compartment_id PASSED [ 23%]
tests/test_oci_client.py::TestOciClientModels::test_command_a_chat PASSED [ 25%]
tests/test_oci_client.py::TestOciClientModels::test_embed_english_v3 PASSED [ 27%]
tests/test_oci_client.py::TestOciClientModels::test_embed_multilingual_v3 PASSED [ 29%]
tests/test_oci_client.py::TestOciClientTransformations (33 unit tests) ALL PASSED [31-100%]

======================= 47 passed in 7.90s ========================

V1 (OciClient) — Command R family, COHERE apiFormat

Test	Model	Result
test_embed	embed-english-v3.0	2x 1024-dim float vectors
test_chat	command-r-08-2024	V1 text response
test_chat_stream	command-r-08-2024	V1 text-generation stream events

V2 (OciClientV2) — Command A family, COHEREV2 apiFormat

Test	Model	Result
test_embed_v2	embed-english-v3.0	dict with float_ key
test_chat_v2	command-a-03-2025	V2 message response
test_chat_stream_v2	command-a-03-2025	V2 content-delta SSE events

Validation tests

Test	What it proves
test_v2_client_rejects_v1_request	OciClientV2 + V1 body → clear ValueError
test_v1_client_rejects_v2_request	OciClient + V2 body → clear ValueError

Fern safety

oci_client.py → .fernignore
lazy_oci_deps.py → manually_maintained/
__init__.py exports → feat(python): add OciClient to additional_init_exports cohere-developer-experience#712

fede-kamel · 2026-03-24T16:14:52Z

@mkozakov Quick summary of where everything stands — all work is done and tested on our side.

3 PRs, all ready for review:

cohere-python#718 (this PR) — OCI client with OciClient (V1) + OciClientV2 (V2), 47 tests passing live against OCI GenAI
cohere-python#698 — embed_stream() for memory-efficient embedding, 9 unit tests + 6 e2e validated via OCI
cohere-developer-experience#712 — one-line Fern config change so OciClient/OciClientV2 survive init.py regeneration

All code is Fern-safe — zero auto-generated files modified across any of these PRs. Let me know if anything needs adjustment.

mkozakov · 2026-04-02T20:27:16Z

Hey @fede-kamel finally had a chance to review. In general this looks good, but not sure why this PR touches the AWS client / removes tests?

Adds OciClient (V1 API) and OciClientV2 (V2 API) for the OCI Generative AI service, following the BedrockClient pattern with httpx event hooks. Authentication: config file, custom profiles, session tokens, direct credentials, instance principal, resource principal. API coverage: embed (all models), chat with streaming (OciClient for Command R family, OciClientV2 for Command A). Lazy-loads oci SDK as an optional dependency; install with `pip install cohere[oci]`.

…profile - README: remove specific model names from Supported APIs and Model Availability sections (per mkozakov review — will go out of date) - tests: default OCI_PROFILE to DEFAULT instead of API_KEY_AUTH

fede-kamel · 2026-04-02T22:24:44Z

Hey @mkozakov — yes, all addressed! Here's the summary:

AWS client / removed tests: Good catch. An earlier iteration of this branch touched the AWS client and removed the AWS unit tests as a side effect. That's all been fully reverted — aws_client.py, cohere_aws/client.py, and test_aws_client_unit.py are back to exactly what's on main. This PR now only touches OCI files.

All 3 inline comments from your review are addressed:

Removed specific model names from the Supported OCI APIs list (won't go stale)
Removed specific model names from the Model Availability section (same reason)
Reverted all AWS client changes (see above)

Branch has been squashed to 2 clean commits:

feat: Add Oracle Cloud Infrastructure (OCI) Generative AI client support
fix: address review feedback — remove stale model names and fix test profile

Tests: 47/47 passing against live OCI GenAI (us-chicago-1).

The "stream" in endpoint check was dead code — both V1 and V2 SDK always route through endpoint "chat" (v1/chat and v2/chat paths). Streaming is reliably signalled via body["stream"], which the SDK always sets. - Drop "stream" in endpoint guard on is_stream and isStream detection - Remove "chat_stream" from action_map, transform, and response branches - Update unit tests to use "chat" endpoint (the only real one)

_current_content_type now returns None for events with no message content (e.g. {"finishReason": "COMPLETE"}). The transition branch in _transform_v2_event is skipped when event_content_type is None, so a finish-only event after a thinking block no longer opens a spurious empty text block before emitting content-end.

…urn, and system message

cursor

Cursor Bugbot has reviewed your changes and found 2 potential issues.

There are 3 total unresolved issues (including 1 from previous review).

… token per-request - transform_request_to_oci now raises ValueError for endpoints other than 'embed' and 'chat' instead of silently returning the untransformed body - Session token auth uses a refreshing wrapper that re-reads the token file before each signing call, so OCI CLI token refreshes are picked up without restarting the client - Add test_unsupported_endpoint_raises to cover the new explicit error - Update test_session_auth_prefers_security_token_signer to expect multi-call behaviour from the refreshing signer

test_session_token_refreshed_on_subsequent_requests writes a real token file, makes two requests with the file updated between them, and asserts that the second signing call uses the new token — verifying the refreshing signer works end-to-end.

fede-kamel · 2026-04-09T14:55:41Z

Is there by chance a response_id you can use? cc: @daniel-cohere

Confirmed from the OCI side — just tested both V1 and V2 responses against the live OCI GenAI inference service:

V2 (Command A, COHEREV2 format): The response includes an id field (e.g. 51122e24-2c71-40e3-bc26-d13db4e89c40), which the code already maps correctly. No response_id or generation_id exists in the schema.

V1 (Command R, COHERE format): The response has no ID fields at all — no id, response_id, generation_id, or request_id. The only ID available is opc-request-id in the HTTP headers, but that's a request-level trace ID (three-part format), not a response/generation identifier.

So the synthetic uuid.uuid4() for V1 is the only option — OCI simply doesn't expose a response-level ID in the V1 chat schema.

#712) Add OCI client exports so they survive Fern regeneration of __init__.py. Follows the same pattern as BedrockClient/SagemakerClient. Related: cohere-ai/cohere-python#718

cursor Bot reviewed Jan 26, 2026

View reviewed changes

Comment thread pyproject.toml Outdated

Comment thread src/cohere/oci_client.py Outdated

fede-kamel force-pushed the feat/oci-client branch from 0e341c1 to fdebc00 Compare January 26, 2026 04:13

cursor Bot reviewed Jan 26, 2026

View reviewed changes

Comment thread src/cohere/oci_client.py

fede-kamel force-pushed the feat/oci-client branch from a284ea8 to d7c7ef6 Compare January 26, 2026 04:42

cursor Bot reviewed Jan 26, 2026

View reviewed changes

Comment thread src/cohere/oci_client.py Outdated

Comment thread src/cohere/oci_client.py Outdated

cursor Bot reviewed Jan 26, 2026

View reviewed changes

Comment thread src/cohere/oci_client.py Outdated

cursor Bot reviewed Feb 4, 2026

View reviewed changes

Comment thread src/cohere/oci_client.py

Comment thread src/cohere/oci_client.py Outdated

Comment thread src/cohere/oci_client.py

Comment thread src/cohere/oci_client.py Outdated

Comment thread src/cohere/oci_client.py Outdated

cursor Bot reviewed Feb 6, 2026

View reviewed changes

Comment thread src/cohere/oci_client.py Outdated

Comment thread src/cohere/oci_client.py

cursor Bot reviewed Feb 6, 2026

View reviewed changes

Comment thread src/cohere/oci_client.py Outdated

fede-kamel force-pushed the feat/oci-client branch from 49f92cc to 8b9c63d Compare February 24, 2026 20:51

cursor Bot reviewed Feb 24, 2026

View reviewed changes

Comment thread src/cohere/oci_client.py Outdated

Comment thread src/cohere/oci_client.py

fede-kamel force-pushed the feat/oci-client branch from 3bf4c54 to b78c63e Compare February 25, 2026 15:40

fede-kamel mentioned this pull request Feb 25, 2026

Feature request: Oracle Cloud Infrastructure (OCI) Generative AI client support #735

Closed

cursor Bot reviewed Feb 25, 2026

View reviewed changes

Comment thread src/cohere/oci_client.py Outdated

Comment thread src/cohere/oci_client.py Outdated

Comment thread src/cohere/oci_client.py Outdated

Comment thread src/cohere/oci_client.py Outdated

Comment thread src/cohere/oci_client.py

sanderland approved these changes Feb 25, 2026

View reviewed changes

fern-support mentioned this pull request Mar 12, 2026

feat: Add Oracle Cloud Infrastructure (OCI) Generative AI client support #737

Closed

cursor Bot reviewed Mar 12, 2026

View reviewed changes

Comment thread src/cohere/oci_client.py Outdated

Comment thread src/cohere/oci_client.py Outdated

fede-kamel force-pushed the feat/oci-client branch from 1119be8 to 12938af Compare March 12, 2026 14:30

cursor Bot reviewed Mar 12, 2026

View reviewed changes

Comment thread src/cohere/oci_client.py Outdated

cursor Bot reviewed Mar 12, 2026

View reviewed changes

Comment thread src/cohere/oci_client.py Outdated

Comment thread src/cohere/oci_client.py

cursor Bot reviewed Mar 12, 2026

View reviewed changes

Comment thread src/cohere/oci_client.py

Comment thread src/cohere/oci_client.py

fede-kamel force-pushed the feat/oci-client branch from fb5c414 to b3fcdee Compare March 14, 2026 23:58

cursor Bot reviewed Mar 15, 2026

View reviewed changes

Comment thread src/cohere/oci_client.py Outdated

Comment thread src/cohere/oci_client.py Outdated

cursor Bot reviewed Mar 15, 2026

View reviewed changes

Comment thread src/cohere/oci_client.py Outdated

Comment thread src/cohere/oci_client.py Outdated

cursor Bot reviewed Mar 15, 2026

View reviewed changes

Comment thread src/cohere/oci_client.py Outdated

Comment thread src/cohere/oci_client.py Outdated

Comment thread src/cohere/oci_client.py Outdated

cursor Bot reviewed Mar 24, 2026

View reviewed changes

Comment thread src/cohere/aws_client.py

Comment thread src/cohere/manually_maintained/cohere_aws/client.py Outdated

Comment thread src/cohere/manually_maintained/streaming.py

mkozakov reviewed Apr 2, 2026

View reviewed changes

Comment thread README.md

Comment thread README.md Outdated

Comment thread src/cohere/aws_client.py

fede-kamel force-pushed the feat/oci-client branch from d48cca4 to d520316 Compare April 2, 2026 22:13

fede-kamel added 2 commits April 2, 2026 18:21

fix: address review feedback — remove stale model names and fix test …

636e76e

…profile - README: remove specific model names from Supported APIs and Model Availability sections (per mkozakov review — will go out of date) - tests: default OCI_PROFILE to DEFAULT instead of API_KEY_AUTH

fede-kamel force-pushed the feat/oci-client branch from a6a996d to 636e76e Compare April 2, 2026 22:22

cursor Bot reviewed Apr 2, 2026

View reviewed changes

Comment thread src/cohere/oci_client.py Outdated

cursor Bot reviewed Apr 2, 2026

View reviewed changes

Comment thread src/cohere/oci_client.py Outdated

fede-kamel added 2 commits April 2, 2026 18:56

test: add integration tests for light models, command-r-plus, multi-t…

81f159b

…urn, and system message

cursor Bot reviewed Apr 2, 2026

View reviewed changes

Comment thread src/cohere/oci_client.py Outdated

Comment thread src/cohere/oci_client.py Outdated

fede-kamel added 3 commits April 2, 2026 19:10

fix: use 'or []' to guard against explicit content=None in V2 messages

2ac5ec4

fede-kamel requested a review from mkozakov April 4, 2026 17:44

daniel-cohere reviewed Apr 8, 2026

View reviewed changes

Comment thread src/cohere/oci_client.py

Merge branch 'main' into feat/oci-client

a06825a

daniel-cohere enabled auto-merge (squash) April 9, 2026 16:30

daniel-cohere approved these changes Apr 9, 2026

View reviewed changes

daniel-cohere merged commit 152dbb1 into cohere-ai:main Apr 9, 2026
1 check passed

daniel-cohere mentioned this pull request Apr 9, 2026

Revert "feat: Add Oracle Cloud Infrastructure (OCI) Generative AI client support" #753

Merged

This was referenced Apr 9, 2026

feat(oci): Add Oracle Cloud Infrastructure (OCI) Generative AI client support #754

Merged

docs: Add native Cohere Python SDK support for OCI Generative AI cohere-ai/cohere-developer-experience#740

Merged

Uh oh!

Conversation

fede-kamel commented Jan 26, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Motivation

Implementation

Features

Architecture

Testing

Documentation

Files Changed

Test Results

Breaking Changes

Checklist

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fede-kamel commented Jan 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fede-kamel commented Feb 24, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fede-kamel commented Feb 26, 2026

Uh oh!

fede-kamel commented Mar 3, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fede-kamel commented Mar 15, 2026

Uh oh!

fede-kamel commented Mar 24, 2026

Final test run — 47/47 passed

V1 (OciClient) — Command R family, COHERE apiFormat

V2 (OciClientV2) — Command A family, COHEREV2 apiFormat

Validation tests

Fern safety

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fede-kamel commented Mar 24, 2026

Uh oh!

mkozakov commented Apr 2, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fede-kamel commented Apr 2, 2026

fede-kamel commented Jan 26, 2026 •

edited by cursor Bot

Loading

fede-kamel commented Jan 26, 2026 •

edited

Loading

fede-kamel commented Apr 9, 2026 •

edited

Loading