Answer on an Anthropic key from the AG2 Bot, not only on an OpenAI one

The AG2 Bot built `OpenAIConfig(model=BOT_MODEL)` whatever the setup screen chose and never read
`BOT_PROVIDER`. The catalogue offers every harness it installs with any model the screen offers, so
AG2 can be picked with an Anthropic key: the desktop then writes `BOT_PROVIDER=anthropic`,
`BOT_MODEL=claude-sonnet-4-5` and an empty `OPENAI_API_KEY`, and every run failed inside the
OpenAI client with `OpenAIError: Missing credentials` before any request was made.

AG2 reaches Anthropic through its own `AnthropicConfig`, from the `anthropic` extra, which reads
`ANTHROPIC_API_KEY` and `ANTHROPIC_BASE_URL` the way the Anthropic SDK does. The harness now picks
that config when the provider is `anthropic` and `OpenAIConfig` otherwise, so an OpenAI key and an
OpenAI-compatible endpoint behave as they did. AG2 has no LiteLLM config to route through, which is
why this is a branch rather than one `provider/model` string as in the LlamaIndex Bot.

The test follows the LlamaIndex Bot's. Before this change the Anthropic key choice fails and the
other two pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Author: 2 people

.github/workflows/ci.yml

@@ -318,6 +318,13 @@ jobs:
           . .venv-langroid/bin/activate
           python -m pip install --requirement agent-langroid/requirements.txt --requirement agent-langroid/requirements-test.txt
           python -m pytest agent-langroid/tests -q
+      - name: AG2 model-choice regression
+        run: |
+          set -euo pipefail
+          python -m venv .venv-ag2
+          . .venv-ag2/bin/activate
+          python -m pip install --requirement agent-ag2/requirements.txt --requirement agent-ag2/requirements-test.txt
+          python -m pytest agent-ag2/tests -q
       - run: bun install --frozen-lockfile
       - run: bun test tests/compose.test.ts
       - run: docker compose --env-file /dev/null --profile harness config --format json >/dev/null

CHANGELOG.md

@@ -32,6 +32,14 @@ builds an OpenAI client for such a model all the same, from the `OPENAI_API_KEY`
 empty when the choice was not OpenAI. The image now installs Langroid's litellm extra and an empty
 OpenAI key is treated as none, so the Bot starts and answers with the Anthropic model chosen.
 
+### The AG2 Bot answers on an Anthropic key
+
+The AG2 Bot built an OpenAI client whatever the setup screen chose, and read `BOT_MODEL` but never
+`BOT_PROVIDER`. Picked with an Anthropic key, every run failed asking for an OpenAI key, because
+Compose writes that one empty when the choice was Anthropic. It now reaches Anthropic through AG2's
+own Anthropic client when that is the provider chosen, and OpenAI or an OpenAI-compatible endpoint
+as before otherwise.
+
 ## 0.0.13
 
 ### Fresh desktop setup installs its runtime before sign-in

agent-ag2/requirements-test.txt

@@ -0,0 +1,2 @@
+httpx==0.28.1
+pytest==9.0.2

agent-ag2/requirements.txt

@@ -1,4 +1,4 @@
-ag2[ag-ui,openai]
+ag2[ag-ui,anthropic,openai]
 fastapi
 python-multipart
 uvicorn[standard]

agent-ag2/src/main.py

@@ -4,16 +4,30 @@
 
 from ag2 import Agent
 from ag2.ag_ui import AGUIStream
-from ag2.config import OpenAIConfig
+from ag2.config import AnthropicConfig, OpenAIConfig
 from fastapi import FastAPI, Request
 from fastapi.responses import JSONResponse
 
 TOKEN_HEADER = "x-openbot-agent-token"
 
+
+def _config() -> AnthropicConfig | OpenAIConfig:
+    """The provider the model screen chose, which AG2 reaches through a config of its own.
+
+    `BOT_PROVIDER` is `anthropic` for an Anthropic key and `openai` otherwise, an OpenAI-compatible
+    endpoint included. Each SDK reads its own key and base URL from the environment.
+    """
+    provider = (os.environ.get("BOT_PROVIDER") or "openai").strip()
+    model = (os.environ.get("BOT_MODEL") or "gpt-4o-mini").strip()
+    if provider == "anthropic":
+        return AnthropicConfig(model=model)
+    return OpenAIConfig(model=model)
+
+
 agent = Agent(
     name="openbot",
     prompt="Answer the question you are asked, briefly and correctly.",
-    config=OpenAIConfig(model=(os.environ.get("BOT_MODEL") or "gpt-4o-mini").strip()),
+    config=_config(),
 )
 stream = AGUIStream(agent)
 

agent-ag2/tests/test_main.py

@@ -0,0 +1,184 @@
+import importlib
+import json
+import socket
+import sys
+import threading
+import time
+from pathlib import Path
+
+import pytest
+import uvicorn
+from fastapi import FastAPI, Request
+from fastapi.responses import JSONResponse, StreamingResponse
+from fastapi.testclient import TestClient
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
+
+TOKEN = "test-token"
+RUN = {
+    "threadId": "thread-1",
+    "runId": "run-1",
+    "state": {},
+    "messages": [{"id": "m1", "role": "user", "content": "Say hello"}],
+    "tools": [],
+    "context": [],
+    "forwardedProps": {},
+}
+
+
+def _sse(events):
+    async def stream():
+        for event in events:
+            yield event
+
+    return StreamingResponse(stream(), media_type="text/event-stream")
+
+
+def _provider_app(seen):
+    app = FastAPI()
+
+    @app.post("/v1/chat/completions")
+    async def openai_chat(request: Request):
+        body = await request.json()
+        seen.append(("openai", body["model"]))
+        if not body.get("stream"):
+            return JSONResponse(
+                {
+                    "id": "c",
+                    "object": "chat.completion",
+                    "created": 0,
+                    "model": body["model"],
+                    "choices": [
+                        {
+                            "index": 0,
+                            "finish_reason": "stop",
+                            "message": {"role": "assistant", "content": "hello"},
+                        }
+                    ],
+                }
+            )
+        chunk = {
+            "id": "c",
+            "object": "chat.completion.chunk",
+            "created": 0,
+            "model": body["model"],
+            "choices": [
+                {
+                    "index": 0,
+                    "delta": {"role": "assistant", "content": "hello"},
+                    "finish_reason": None,
+                }
+            ],
+        }
+        done = {**chunk, "choices": [{"index": 0, "delta": {}, "finish_reason": "stop"}]}
+        return _sse(
+            [f"data: {json.dumps(chunk)}\n\n", f"data: {json.dumps(done)}\n\n", "data: [DONE]\n\n"]
+        )
+
+    @app.post("/v1/messages")
+    async def anthropic_messages(request: Request):
+        body = await request.json()
+        seen.append(("anthropic", body["model"]))
+        message = {
+            "id": "msg",
+            "type": "message",
+            "role": "assistant",
+            "model": body["model"],
+            "stop_sequence": None,
+        }
+        if not body.get("stream"):
+            return JSONResponse(
+                {
+                    **message,
+                    "content": [{"type": "text", "text": "hello"}],
+                    "stop_reason": "end_turn",
+                    "usage": {"input_tokens": 1, "output_tokens": 1},
+                }
+            )
+        events = [
+            ("message_start", {"type": "message_start", "message": {**message, "content": [], "stop_reason": None, "usage": {"input_tokens": 1, "output_tokens": 0}}}),
+            ("content_block_start", {"type": "content_block_start", "index": 0, "content_block": {"type": "text", "text": ""}}),
+            ("content_block_delta", {"type": "content_block_delta", "index": 0, "delta": {"type": "text_delta", "text": "hello"}}),
+            ("content_block_stop", {"type": "content_block_stop", "index": 0}),
+            ("message_delta", {"type": "message_delta", "delta": {"stop_reason": "end_turn", "stop_sequence": None}, "usage": {"output_tokens": 1}}),
+            ("message_stop", {"type": "message_stop"}),
+        ]
+        return _sse([f"event: {name}\ndata: {json.dumps(data)}\n\n" for name, data in events])
+
+    return app
+
+
+@pytest.fixture
+def provider():
+    seen = []
+    with socket.socket() as probe:
+        probe.bind(("127.0.0.1", 0))
+        port = probe.getsockname()[1]
+    server = uvicorn.Server(
+        uvicorn.Config(_provider_app(seen), host="127.0.0.1", port=port, log_level="error")
+    )
+    thread = threading.Thread(target=server.run, daemon=True)
+    thread.start()
+    deadline = time.monotonic() + 10
+    while not server.started and time.monotonic() < deadline:
+        time.sleep(0.01)
+    yield f"http://127.0.0.1:{port}", seen
+    server.should_exit = True
+    thread.join(timeout=10)
+
+
+CHOICES = {
+    "an Anthropic key": (
+        lambda base: {
+            "BOT_PROVIDER": "anthropic",
+            "BOT_MODEL": "claude-sonnet-4-5",
+            "ANTHROPIC_API_KEY": "test-key",
+            "ANTHROPIC_BASE_URL": base,
+            "OPENAI_API_KEY": "",
+            "OPENAI_BASE_URL": "",
+        },
+        ("anthropic", "claude-sonnet-4-5"),
+    ),
+    "an OpenAI-compatible endpoint": (
+        lambda base: {
+            "BOT_PROVIDER": "",
+            "BOT_MODEL": "local-model",
+            "OPENAI_API_KEY": "no-key-needed",
+            "OPENAI_BASE_URL": f"{base}/v1",
+            "ANTHROPIC_API_KEY": "",
+        },
+        ("openai", "local-model"),
+    ),
+    "an OpenAI key": (
+        lambda base: {
+            "BOT_PROVIDER": "",
+            "BOT_MODEL": "gpt-5.5",
+            "OPENAI_API_KEY": "test-key",
+            "OPENAI_BASE_URL": f"{base}/v1",
+            "ANTHROPIC_API_KEY": "",
+        },
+        ("openai", "gpt-5.5"),
+    ),
+}
+
+
+@pytest.mark.parametrize("choice", list(CHOICES))
+def test_a_run_reaches_the_model_the_setup_screen_chose(monkeypatch, provider, choice):
+    base, seen = provider
+    environment, expected = CHOICES[choice]
+    monkeypatch.delenv("OPENAI_API_BASE", raising=False)
+    monkeypatch.setenv("MANAGED_AGENT_TOKEN", TOKEN)
+    for key, value in environment(base).items():
+        monkeypatch.setenv(key, value)
+
+    from src import main
+
+    main = importlib.reload(main)
+    response = TestClient(main.app).post(
+        "/", json=RUN, headers={"x-openbot-agent-token": TOKEN}
+    )
+
+    assert response.status_code == 200
+    assert '"RUN_FINISHED"' in response.text
+    assert '"RUN_ERROR"' not in response.text
+    assert seen == [expected]