Skip to content

feat: add OrcaRouter as a named backend - #183

Open
zsanig22-dotcom wants to merge 1 commit into
alexzhang13:mainfrom
zsanig22-dotcom:feat/orcarouter-backend
Open

zsanig22-dotcom wants to merge 1 commit into
alexzhang13:mainfrom
zsanig22-dotcom:feat/orcarouter-backend

Conversation

@zsanig22-dotcom

Copy link
Copy Markdown

Overview

RLM's whole pitch is swapping llm.completion(prompt, model) for a recursive rlm.completion(prompt, model) that can programmatically examine, decompose, and recursively call itself over near-infinite contexts. When you're chaining sub-LM calls that way, every hop goes through your model provider — and the way you reach those models shouldn't force you to pick a single vendor or a bare custom base_url.

This PR adds orcarouter as a first-class backend, mirroring how the existing openrouter backend is wired in rlm/clients/__init__.py and rlm/clients/openai.py.

What's included

  • rlm/clients/__init__.py: new orcarouter branch in get_client() that dispatches to OpenAIClient with base_url defaulting to https://api.orcarouter.ai/v1 — exactly the same pattern as the openrouter and vercel branches.
  • rlm/clients/openai.py: auto-loads ORCAROUTER_API_KEY when api_key is omitted and base_url matches the OrcaRouter endpoint, so the key is picked up from the environment like OPENROUTER_API_KEY.
  • rlm/core/types.py: "orcarouter" added to the ClientBackend literal.
  • Docs: README (Model Providers), docs/api/rlm.md (backend literal + kwargs table), docs/getting-started.md, and the docs site's backends and api pages.
  • Tests: dispatch test in tests/test_imports.py asserting the orcarouter backend resolves to OpenAIClient with the correct base URL.

Usage looks like the existing OpenRouter snippet:

rlm = RLM(
    backend="orcarouter",
    backend_kwargs={
        "api_key": os.getenv("ORCAROUTER_API_KEY"),
        "model_name": "orcarouter/auto",  # or vendor/model, e.g. anthropic/claude-opus-4.8
    },
)

Why OrcaRouter

OrcaRouter is an OpenAI-compatible AI gateway built for both models and agents. Like OpenRouter, it exposes a provider/model namespace across many models — but it also combines adaptive routing, automatic failover, zero-markup inference, observability, guardrails, and agent-tool governance behind the same endpoint. Adding orcarouter as a first-class provider means RLM users can use that stack directly, without treating OrcaRouter as an anonymous custom base URL. It also runs gateway-level, zero-trust security for AI agents on the same endpoint — screening every prompt/response and governing every tool call on a default-deny basis, with no application code changes.

Verification

  • ruff check . and ruff format --check . pass (0.14.10).
  • pytest tests/ --ignore=tests/repl/test_modal_repl.py --ignore=tests/clients/ — 251 passed, including the new test_orcarouter_backend_dispatch.
  • Live-tested against https://api.orcarouter.ai/v1 through the new backend path: client.completion("Reply with exactly: ORCAOK") returned ORCAOK (200, token usage tracked).

Discord: discord.gg/YEubt8enRA · X: https://x.com/OrcaRouter

I'm an engineer on the OrcaRouter team.

Add `orcarouter` as a first-class backend mirroring the existing
`openrouter` wiring: it dispatches to OpenAIClient with a default
base_url of https://api.orcarouter.ai/v1 and auto-loads
ORCAROUTER_API_KEY when no api_key is passed.

Also updates the ClientBackend literal, README, and the docs
(backends, api reference, and getting-started).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant