Skip to content

Commit 482bcf2

Browse files
feat: add OrcaRouter as a named backend
Add `orcarouter` as a first-class backend mirroring the existing `openrouter` wiring: it dispatches to OpenAIClient with a default base_url of https://api.orcarouter.ai/v1 and auto-loads ORCAROUTER_API_KEY when no api_key is passed. Also updates the ClientBackend literal, README, and the docs (backends, api reference, and getting-started).
1 parent 854e688 commit 482bcf2

9 files changed

Lines changed: 53 additions & 3 deletions

File tree

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -127,7 +127,7 @@ export PRIME_API_KEY=...
127127

128128

129129
### Model Providers
130-
We currently support most major clients (OpenAI, Anthropic), as well as the router platforms (OpenRouter, Portkey). For local models, we recommend using vLLM (which interfaces with the [OpenAI client](https://github.com/alexzhang13/rlm/blob/main/rlm/clients/openai.py)). To view or add support for more clients, start by looking at [`rlm/clients/`](https://github.com/alexzhang13/rlm/tree/main/rlm/clients).
130+
We currently support most major clients (OpenAI, Anthropic), as well as the router platforms (OpenRouter, Portkey, OrcaRouter). For local models, we recommend using vLLM (which interfaces with the [OpenAI client](https://github.com/alexzhang13/rlm/blob/main/rlm/clients/openai.py)). To view or add support for more clients, start by looking at [`rlm/clients/`](https://github.com/alexzhang13/rlm/tree/main/rlm/clients).
131131

132132
## Training
133133
We provide a simple RL training harness for training RLMs used in this repo (specifically the `local` REPL). The implementation uses no sandboxes for simplicity and slots easily your use case, but an ideal setup would use sandboxes for safety. Training logic is isolated to the [`training/`](https://github.com/alexzhang13/rlm/tree/main/training) folder, which exposes `rlm.RLM` as a [`verifiers`](https://github.com/willccbb/verifiers) `Environment` and plugs straight into [`prime-rl`](https://github.com/PrimeIntellect-ai/prime-rl). See the [training README](https://github.com/alexzhang13/rlm/tree/main/training#readme) for the launch command. The harness uses subprocess-isolated local REPL execution (no cloud sandboxes), matching the `local` environment above.

‎docs/api/rlm.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ RLM(
7171
#### `backend`
7272
{: .no_toc }
7373

74-
**Type:** `Literal["openai", "portkey", "openrouter", "vllm", "anthropic"]`
74+
**Type:** `Literal["openai", "portkey", "openrouter", "orcarouter", "vllm", "anthropic"]`
7575
**Default:** `"openai"`
7676

7777
The LM provider backend to use for the root model.
@@ -103,6 +103,7 @@ Configuration passed to the LM client. Required fields vary by backend:
103103
| `anthropic` | `model_name` | `api_key` |
104104
| `portkey` | `model_name`, `api_key` | `base_url` |
105105
| `openrouter` | `model_name` | `api_key` |
106+
| `orcarouter` | `model_name` | `api_key` |
106107
| `vllm` | `model_name`, `base_url` | — |
107108

108109

‎docs/getting-started.md‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -317,6 +317,18 @@ rlm = RLM(
317317
)
318318
```
319319

320+
### OrcaRouter
321+
322+
```python
323+
rlm = RLM(
324+
backend="orcarouter",
325+
backend_kwargs={
326+
"api_key": os.getenv("ORCAROUTER_API_KEY"),
327+
"model_name": "orcarouter/auto", # Adaptive routing, or use vendor/model
328+
},
329+
)
330+
```
331+
320332
### vLLM (Local)
321333

322334
```python

‎docs/src/app/api/page.tsx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,7 @@ print(result.response)`} />
6161
[<code key="2" className="text-sm">"anthropic"</code>, "Anthropic API"],
6262
[<code key="3" className="text-sm">"portkey"</code>, "Portkey AI gateway"],
6363
[<code key="4" className="text-sm">"openrouter"</code>, "OpenRouter"],
64+
[<code key="5" className="text-sm">"orcarouter"</code>, "OrcaRouter"],
6465

6566
[<code key="6" className="text-sm">"vllm"</code>, "Local vLLM server"],
6667
]}

‎docs/src/app/backends/page.tsx‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,20 @@ export default function BackendsPage() {
6464

6565
<hr className="my-8 border-border" />
6666

67+
<h2 className="text-2xl font-semibold mb-4">OrcaRouter</h2>
68+
<p className="text-muted-foreground mb-4">
69+
<a href="https://www.orcarouter.ai" className="text-primary underline font-medium" target="_blank" rel="noopener noreferrer">OrcaRouter</a> is an OpenAI-compatible AI gateway that routes requests at upstream cost across 150+ models from OpenAI, Anthropic, Google Gemini, DeepSeek, and more — with adaptive routing, automatic failover, and observability.
70+
</p>
71+
<CodeBlock code={`rlm = RLM(
72+
backend="orcarouter",
73+
backend_kwargs={
74+
"api_key": os.getenv("ORCAROUTER_API_KEY"),
75+
"model_name": "anthropic/claude-opus-4.8", # Format: vendor/model, or orcarouter/auto
76+
},
77+
)`} />
78+
79+
<hr className="my-8 border-border" />
80+
6781
<h2 className="text-2xl font-semibold mb-4">vLLM (Local)</h2>
6882
<p className="text-muted-foreground mb-4">Local model serving.</p>
6983
<CodeBlock language="bash" code={`# Start vLLM server

‎rlm/clients/__init__.py‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,11 @@ def get_client(
3636

3737
backend_kwargs.setdefault("base_url", "https://openrouter.ai/api/v1")
3838
return OpenAIClient(**backend_kwargs)
39+
elif backend == "orcarouter":
40+
from rlm.clients.openai import OpenAIClient
41+
42+
backend_kwargs.setdefault("base_url", "https://api.orcarouter.ai/v1")
43+
return OpenAIClient(**backend_kwargs)
3944
elif backend == "vercel":
4045
from rlm.clients.openai import OpenAIClient
4146

@@ -55,5 +60,5 @@ def get_client(
5560
return AzureOpenAIClient(**backend_kwargs)
5661
else:
5762
raise ValueError(
58-
f"Unknown backend: {backend}. Supported backends: ['openai', 'vllm', 'portkey', 'openrouter', 'anthropic', 'azure_openai', 'gemini', 'vercel']"
63+
f"Unknown backend: {backend}. Supported backends: ['openai', 'vllm', 'portkey', 'openrouter', 'orcarouter', 'anthropic', 'azure_openai', 'gemini', 'vercel']"
5964
)

‎rlm/clients/openai.py‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@
1515
DEFAULT_OPENROUTER_API_KEY = os.getenv("OPENROUTER_API_KEY")
1616
DEFAULT_VERCEL_API_KEY = os.getenv("AI_GATEWAY_API_KEY")
1717
DEFAULT_PRIME_API_KEY = os.getenv("PRIME_API_KEY")
18+
DEFAULT_ORCAROUTER_API_KEY = os.getenv("ORCAROUTER_API_KEY")
1819
DEFAULT_PRIME_INTELLECT_BASE_URL = "https://api.pinference.ai/api/v1/"
1920

2021

@@ -66,6 +67,8 @@ def __init__(
6667
api_key = DEFAULT_OPENAI_API_KEY
6768
elif base_url == "https://openrouter.ai/api/v1":
6869
api_key = DEFAULT_OPENROUTER_API_KEY
70+
elif base_url == "https://api.orcarouter.ai/v1":
71+
api_key = DEFAULT_ORCAROUTER_API_KEY
6972
elif base_url == "https://ai-gateway.vercel.sh/v1":
7073
api_key = DEFAULT_VERCEL_API_KEY
7174
elif base_url == DEFAULT_PRIME_INTELLECT_BASE_URL:

‎rlm/core/types.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66
"openai",
77
"portkey",
88
"openrouter",
9+
"orcarouter",
910
"vercel",
1011
"vllm",
1112
"anthropic",

‎tests/test_imports.py‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,19 @@ def test_get_client_function(self):
7373

7474
assert callable(get_client)
7575

76+
def test_orcarouter_backend_dispatch(self):
77+
"""Test that get_client routes the orcarouter backend to OpenAIClient."""
78+
pytest.importorskip("openai")
79+
from rlm.clients import get_client
80+
from rlm.clients.openai import OpenAIClient
81+
82+
client = get_client(
83+
"orcarouter",
84+
{"api_key": "sk-test", "model_name": "orcarouter/auto"},
85+
)
86+
assert isinstance(client, OpenAIClient)
87+
assert client.base_url == "https://api.orcarouter.ai/v1"
88+
7689

7790
class TestCoreImports:
7891
"""Test core module imports."""

0 commit comments

Comments
 (0)