Merge pull request #205 from XiaoHuo888-hue/add-orcarouter-provider

feat(llm): add OrcaRouter as a named OpenAI-compatible gateway

Author: LauraGPT

README.md

@@ -148,6 +148,14 @@ Follow the guide below to explore LLM based clipping:
 
 <img src="docs/images/LLM_guide.png" width=360/>
 
+#### Using OrcaRouter as your LLM gateway (optional)
+
+Besides the transcript-based LLMs above, FunClip can route LLM-assisted clipping through [OrcaRouter](https://www.orcarouter.ai), an OpenAI-compatible smart-routing gateway. Select any `orcarouter/` model in the **LLM Model Name** dropdown (`orcarouter/auto` routes each request to the best model for the task), paste an OrcaRouter API key in the **APIKEY** box, and click 'LLM Inference' — FunClip sends the transcript and prompts to `https://api.orcarouter.ai/v1/chat/completions`, and the returned segments work with the existing 'AI Clip' button unchanged.
+
+OrcaRouter exposes one endpoint for all frontier and open-weight models, so you can switch routing targets without changing FunClip. It also offers gateway-level, zero-trust controls for AI agents on the same endpoint. Enforcement follows the Guardrail or Firewall policy attached to the relevant scope for prompts, responses, and tool calls that traverse the gateway; select the documented `tight` posture to enable default-deny behavior. Enforcement is opt-in — this integration supplies only the base URL, API key, and model, and does not attach a policy by itself. Apply a policy via the [security quickstart](https://docs.orcarouter.ai/security/concepts/quickstart); see [Guardrails](https://docs.orcarouter.ai/features/guardrails#scoping-and-the-workspace-default) and [Firewall](https://docs.orcarouter.ai/features/firewall#scoping-and-resolution) for how coverage is scoped.
+
+Set `ORCAROUTER_API_KEY` (and optionally `ORCAROUTER_API_BASE`, which defaults to `https://api.orcarouter.ai/v1`) instead of pasting the key into the UI if you prefer environment-based configuration. A key is available at https://www.orcarouter.ai.
+
 #### Content-aware clipping with TwelveLabs Pegasus (optional)
 
 Besides the transcript-based LLMs above, FunClip can optionally use [TwelveLabs](https://twelvelabs.io) Pegasus, a video understanding model that reasons over the actual video (visuals + audio) rather than only the ASR transcript. This helps pick highlight segments even when the transcript alone is ambiguous (e.g. action, scene changes, on-screen events). To use it, select the `pegasus1.5` model name, paste your TwelveLabs API key, upload a video, and click 'LLM Inference' — Pegasus returns segments in the same `N. [start-end] text` format, so the existing 'AI Clip' button works unchanged. It needs `pip install twelvelabs`, and a free API key is available at https://twelvelabs.io.

README_zh.md

@@ -148,6 +148,14 @@ python funclip/launch.py
 
 <img src="docs/images/LLM_guide.png" width=360/>
 
+#### 使用 OrcaRouter 作为 LLM 网关（可选）
+
+除基于字幕的 LLM 外，FunClip 也可以将 LLM 智能裁剪路由到 [OrcaRouter](https://www.orcarouter.ai)——一个 OpenAI 兼容的智能路由网关。在 **LLM Model Name** 下拉框选择任意 `orcarouter/` 模型（`orcarouter/auto` 会自动为任务选择最佳模型），在 **APIKEY** 输入框粘贴 OrcaRouter API key，点击“LLM推理”——FunClip 会把字幕与 prompt 发送到 `https://api.orcarouter.ai/v1/chat/completions`，返回的分段与现有“AI Clip”按钮完全兼容。
+
+OrcaRouter 用单一端点接入所有前沿与开源模型，无需修改 FunClip 即可切换路由目标。它还可以在同一端点上为 AI agent 提供网关级零信任控制。对经网关转发的 prompt、response 与工具调用，实际执行方式由相关作用域附加的 Guardrail 或 Firewall 策略决定；选择文档中的 `tight` posture 才会启用默认拒绝（default-deny）。该防护为可选开启：本集成只提供 base URL、API key 与模型，本身不会附加策略。请按 [security quickstart](https://docs.orcarouter.ai/security/concepts/quickstart) 应用策略；覆盖范围详见 [Guardrails](https://docs.orcarouter.ai/features/guardrails#scoping-and-the-workspace-default) 与 [Firewall](https://docs.orcarouter.ai/features/firewall#scoping-and-resolution)。
+
+也可以不填 UI，而是设置 `ORCAROUTER_API_KEY` 环境变量（可选 `ORCAROUTER_API_BASE`，默认为 `https://api.orcarouter.ai/v1`）。Key 可在 https://www.orcarouter.ai 获取。
+
 ### B.通过命令行调用使用FunClip的相关功能
 ```shell
 # 下载下面命令用到的示例视频

funclip/launch.py

@@ -163,7 +163,7 @@ def video_clip_addsub(dest_text, video_spk_input, start_ost, end_ost, state, out
             )
 
     def llm_inference(system_content, user_content, srt_text, model, apikey, video_input=None):
-        SUPPORT_LLM_PREFIX = ['litellm', 'qwen', 'gpt', 'g4f', 'moonshot', 'deepseek', 'atlascloud', 'minimax', 'pegasus']
+        SUPPORT_LLM_PREFIX = ['litellm', 'qwen', 'gpt', 'g4f', 'moonshot', 'deepseek', 'atlascloud', 'minimax', 'orcarouter', 'pegasus']
         if model.startswith('litellm/'):
             return litellm_call(apikey, model, user_content+'\n'+srt_text, system_content)
         if model.startswith('pegasus'):
@@ -175,7 +175,7 @@ def llm_inference(system_content, user_content, srt_text, model, apikey, video_i
             return call_twelvelabs_pegasus(apikey, video_input, model=model, prompt=system_content)
         if model.startswith('qwen'):
             return call_qwen_model(apikey, model, user_content+'\n'+srt_text, system_content)
-        if model.startswith('gpt') or model.startswith('moonshot') or model.startswith('deepseek') or model.startswith('atlascloud/') or model.startswith('minimax/'):
+        if model.startswith('gpt') or model.startswith('moonshot') or model.startswith('deepseek') or model.startswith('atlascloud/') or model.startswith('minimax/') or model.startswith('orcarouter/'):
             return openai_call(apikey, model, user_content+'\n'+srt_text, system_content)
         elif model.startswith('g4f'):
             model = "-".join(model.split('-')[1:])
@@ -282,6 +282,10 @@ def AI_clip_subti(LLM_res, dest_text, video_spk_input, start_ost, end_ost, video
                                              "minimax/MiniMax-M3",
                                              "minimax/MiniMax-M2.7",
                                              "minimax/MiniMax-M2.7-highspeed",
+                                             "orcarouter/auto",
+                                             "orcarouter/fusion",
+                                             "orcarouter/fusion-flash",
+                                             "orcarouter/fusion-mini",
                                              "pegasus1.5"],
                                     value="deepseek-chat",
                                     label="LLM Model Name",

funclip/llm/openai_api.py

@@ -12,6 +12,13 @@
 MINIMAX_API_BASE_CN = "https://api.minimaxi.com/v1"
 MINIMAX_MODEL_PREFIX = "minimax/"
 
+# OrcaRouter is an OpenAI-compatible smart-routing gateway: one chat
+# completions endpoint that routes each request to the best model for the
+# task. Model IDs carry an `orcarouter/` prefix (e.g. `orcarouter/auto`) and
+# must be sent to the gateway verbatim — a bare `auto` is not routable.
+ORCAROUTER_API_BASE = "https://api.orcarouter.ai/v1"
+ORCAROUTER_MODEL_PREFIX = "orcarouter/"
+
 
 def _resolve_model_config(model):
     base_url = None
@@ -37,6 +44,15 @@ def _resolve_model_config(model):
         if not base_url:
             base_url = MINIMAX_API_BASE
         api_key_env = "MINIMAX_API_KEY"
+    elif model.startswith(ORCAROUTER_MODEL_PREFIX):
+        if len(model) <= len(ORCAROUTER_MODEL_PREFIX):
+            raise ValueError(
+                "Model name is empty after stripping orcarouter/ prefix"
+            )
+        base_url = os.environ.get("ORCAROUTER_API_BASE", ORCAROUTER_API_BASE).strip()
+        if not base_url:
+            base_url = ORCAROUTER_API_BASE
+        api_key_env = "ORCAROUTER_API_KEY"
     elif model.startswith("deepseek"):
         base_url = "https://api.deepseek.com"
     elif model.startswith("gpt-3.5-turbo"):

tests/test_orcarouter_api.py

@@ -0,0 +1,93 @@
+"""Tests for OrcaRouter routing through the OpenAI-compatible client."""
+
+import os
+import unittest
+from unittest.mock import MagicMock, patch
+
+from funclip.llm.openai_api import (
+    ORCAROUTER_API_BASE,
+    openai_call,
+)
+
+
+def _mock_completion(content="ok"):
+    completion = MagicMock()
+    completion.choices = [MagicMock()]
+    completion.choices[0].message.content = content
+    return completion
+
+
+class TestOrcaRouterRouting(unittest.TestCase):
+    def test_orcarouter_prefix_uses_gateway_base_url(self):
+        client = MagicMock()
+        client.chat.completions.create.return_value = _mock_completion("clip plan")
+
+        with patch("funclip.llm.openai_api.OpenAI", return_value=client) as openai_cls:
+            result = openai_call(
+                "orca-key",
+                "orcarouter/auto",
+                "subtitle text",
+                "find highlights",
+            )
+
+        self.assertEqual(result, "clip plan")
+        openai_cls.assert_called_once_with(
+            api_key="orca-key",
+            base_url=ORCAROUTER_API_BASE,
+        )
+        # OrcaRouter is a multi-provider gateway: the model ID must keep the
+        # `orcarouter/` prefix so the router knows which namespace to route to.
+        call_kwargs = client.chat.completions.create.call_args[1]
+        self.assertEqual(call_kwargs["model"], "orcarouter/auto")
+
+    def test_orcarouter_api_key_falls_back_to_env(self):
+        client = MagicMock()
+        client.chat.completions.create.return_value = _mock_completion()
+
+        with patch.dict(os.environ, {"ORCAROUTER_API_KEY": "env-orca-key"}, clear=False):
+            with patch("funclip.llm.openai_api.OpenAI", return_value=client) as openai_cls:
+                openai_call("", "orcarouter/auto", "text")
+
+        openai_cls.assert_called_once_with(
+            api_key="env-orca-key",
+            base_url=ORCAROUTER_API_BASE,
+        )
+        call_kwargs = client.chat.completions.create.call_args[1]
+        self.assertEqual(call_kwargs["model"], "orcarouter/auto")
+
+    def test_orcarouter_api_base_env_overrides(self):
+        client = MagicMock()
+        client.chat.completions.create.return_value = _mock_completion()
+
+        with patch.dict(
+            os.environ,
+            {"ORCAROUTER_API_BASE": "https://gateway.example.com/v1"},
+            clear=False,
+        ):
+            with patch("funclip.llm.openai_api.OpenAI", return_value=client) as openai_cls:
+                openai_call("orca-key", "orcarouter/fusion", "text")
+
+        openai_cls.assert_called_once_with(
+            api_key="orca-key",
+            base_url="https://gateway.example.com/v1",
+        )
+
+    def test_empty_orcarouter_model_raises(self):
+        with self.assertRaises(ValueError):
+            openai_call("key", "orcarouter/", "text")
+
+    def test_missing_orcarouter_key_does_not_fall_back_to_openai_key(self):
+        with patch.dict(
+            os.environ,
+            {"OPENAI_API_KEY": "openai-only-key"},
+            clear=True,
+        ):
+            with patch("funclip.llm.openai_api.OpenAI") as openai_cls:
+                with self.assertRaisesRegex(ValueError, "ORCAROUTER_API_KEY"):
+                    openai_call("", "orcarouter/auto", "text")
+
+        openai_cls.assert_not_called()
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/test_orcarouter_launch_integration.py

@@ -0,0 +1,74 @@
+"""Regression tests for OrcaRouter choices and prompt routing in the launcher."""
+
+import ast
+import unittest
+from pathlib import Path
+
+
+LAUNCH_PATH = Path(__file__).resolve().parents[1] / "funclip" / "launch.py"
+
+
+class TestOrcaRouterLaunchIntegration(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls):
+        cls.tree = ast.parse(LAUNCH_PATH.read_text(encoding="utf-8"))
+
+    def test_openai_compatible_route_handles_orcarouter_prefix(self):
+        llm_inference = next(
+            node
+            for node in ast.walk(self.tree)
+            if isinstance(node, ast.FunctionDef) and node.name == "llm_inference"
+        )
+        openai_call = next(
+            node
+            for node in ast.walk(llm_inference)
+            if isinstance(node, ast.Call)
+            and isinstance(node.func, ast.Name)
+            and node.func.id == "openai_call"
+        )
+        # The openai_call dispatch branch must also cover orcarouter/ models.
+        dispatch_condition = next(
+            node
+            for node in ast.walk(llm_inference)
+            if isinstance(node, ast.If) and isinstance(node.test, ast.BoolOp)
+        )
+        self.assertIn("orcarouter/", ast.unparse(dispatch_condition.test))
+        self.assertEqual(ast.unparse(openai_call.args[2]), "user_content + '\\n' + srt_text")
+        self.assertEqual(ast.unparse(openai_call.args[3]), "system_content")
+
+    def test_support_prefix_list_includes_orcarouter(self):
+        llm_inference = next(
+            node
+            for node in ast.walk(self.tree)
+            if isinstance(node, ast.FunctionDef) and node.name == "llm_inference"
+        )
+        assigned = [
+            node
+            for node in ast.walk(llm_inference)
+            if isinstance(node, ast.Assign) and node.targets[0].id == "SUPPORT_LLM_PREFIX"
+        ]
+        self.assertEqual(len(assigned), 1)
+        prefix_list = assigned[0].value
+        self.assertIsInstance(prefix_list, ast.List)
+        prefixes = {
+            node.value
+            for node in ast.walk(prefix_list)
+            if isinstance(node, ast.Constant) and isinstance(node.value, str)
+        }
+        self.assertIn("orcarouter", prefixes)
+
+    def test_dropdown_lists_orcarouter_gateway_models(self):
+        string_literals = {
+            node.value
+            for node in ast.walk(self.tree)
+            if isinstance(node, ast.Constant) and isinstance(node.value, str)
+        }
+
+        self.assertIn("orcarouter/auto", string_literals)
+        self.assertIn("orcarouter/fusion", string_literals)
+        self.assertIn("orcarouter/fusion-flash", string_literals)
+        self.assertIn("orcarouter/fusion-mini", string_literals)
+
+
+if __name__ == "__main__":
+    unittest.main()