Skip to content

Commit e240ea8

Browse files
authored
Pick the model and thinking effort for every Bot, from the provider's live list (#1044)
* Pick a model and thinking effort for every built-in Bot, from the provider's live list The model list is read from whatever serves the deployment's model (OpenAI, Anthropic, Google and xAI directly, any OpenAI-compatible endpoint, or a ChatGPT or Claude plan) and kept current in the background: read at start-up, every six hours, and at once when a provider refuses a model as unknown or unsupported. How hard each model can think comes from the provider's own listing where it says (Anthropic, xAI, the ChatGPT plan) and from models.dev otherwise. The default is the middle of the provider's current line by price, never a configured model the credential cannot use. A pick is now a model and an effort per person and conversation. The effort reaches the model call on every path: the AI SDK's provider options for built-in Bots (reasoning.effort, reasoning_effort, output_config.effort), and forwardedProps.openbotModel for a plan's harness and for the harness picked in setup when it is one of OpenBot's own. A ChatGPT plan's harness is told a model the plan lists, so it no longer runs the gpt-5.5 the plan refuses. One person on their own machine now has picking on by default, as the personal edition already did. * LangGraph AG-UI harness: answer on the server's model, and only on models a ChatGPT plan offers On a ChatGPT plan with Compose's default BOT_MODEL=gpt-5.5, every message failed with "The 'gpt-5.5' model is not supported when using Codex with a ChatGPT account", because the harness sent whatever model the environment named. - Read forwardedProps.openbotModel ({model, reasoningEffort}) on every run, keep it out of graph state, and build the model from it: reasoning.effort on the Codex backend, the Responses API with reasoning.effort for OpenAI models that take tools only there (the shared requiresResponsesApi rule, ported), reasoning_effort on Chat Completions and compatible endpoints, output_config.effort on Anthropic, thinking_level on Google. No effort is sent when the run carries none. - Add GET /models for a signed-in plan: the plan's own Codex listing (chatgpt.com/backend-api/codex/models), visible models only, in Codex's priority order, as {object: "list", data: [{id, label, efforts, default_effort, priority}]}. Cached for six hours, the last good listing kept when a refresh fails, and read again early after a model refusal. - With no server model, a plan answers on the configured model only when its listing offers it, otherwise on the plan's default; with no listing, as before. - A provider refusal now ends the run with RUN_ERROR in the provider's own words (including the Codex backend's {"detail": ...} body, which the OpenAI SDK leaves out of its message), instead of raising out of the stream after RUN_STARTED. - Hold google-genai on 2.x, as in #1041: 3.0.0 moved generateContent to /v1. * Claude Agent SDK harness: answer on the server's model and effort Read forwardedProps.openbotModel on both paths (the /model endpoint for built-in Bots on a Claude plan, and / when this harness is the Bot) and set ClaudeAgentOptions.model and .effort from it, after the upstream merge so the server's choice wins. The model endpoint still drops every other forwarded prop. Efforts Claude Code does not take are not sent. Every query builds its options per run, so a choice changed between runs starts the next query on it. A tool result continues the query that asked for it; when it arrives with another model, the running query is moved to it with ClaudeSDKClient.set_model. The SDK has no live effort change, so a changed effort applies from the next query. * Pick a thinking effort beside the model in the composer The composer's brain icon becomes a text button naming the model and how hard it thinks, "GPT-6.1 Terra · High", the way VS Code's chat does. Its popup keeps the model search and adds a Thinking group for the model in use, listing the efforts it accepts with its own default marked. A model whose default is unknown gets a Default row first, and a model that takes no effort shows no group at all. The efforts sit in the same list as the models, pinned under them, so one keyboard model covers both and a screen reader hears them as options in a labelled group. A listbox option cannot hold a submenu arrow, which is why this is not VS Code's submenu. A conversation's pick is now a model and an effort, read and saved as { model, effort } on /api/channels/:id/model. Each default is saved as null, so it follows the model. Switching model keeps the effort when the new model accepts it and drops it otherwise, so the server is never asked for an effort it would refuse. A new conversation saves both before its first message runs, with the same discard-and-keep-the-draft failure as before. Admin, Providers names whoever actually serves the models by the new source field (OpenAI, Anthropic, Google, xAI, an OpenAI-compatible endpoint, a ChatGPT or Claude plan), with a vendor's mark only where one is vendored. The allowlist still uses provider/model. * Log the model and effort each run is sent One run-model line per run, with the Bot, the run id, the model and the effort, so an operator can see what a conversation actually ran on. Ids only: never the prompt, the person or a credential. * Keep the model list's refusal signal light, list sign-in models through the proxy, and document it The stream observers that report a refused model are imported by the browser app's type check, so the signal moves to a module with no imports instead of pulling in the whole model list. Google and xAI sign-ins list their models through the local proxy, tested on their captured listings. The changelog and configuration docs describe the model list, its refresh and its two new settings. * Mark Google, xAI and Claude, one Default badge, and tell a provider out of credit as that Google, xAI and Claude marks come from @lobehub/icons-static-svg 1.97.0, the set the OpenAI and Anthropic marks already come from, so no model row or trigger starts on an empty gap; a model whose maker has none gets the app's plain model icon. The Default tag is the app's own Badge on every row, so it no longer disappears on the highlighted one. A provider out of credit or at its spending limit is now told as that, with its name and its own words, by built-in Bots (instead of xAI's bare "Forbidden" or OpenAI's "Failed after 3 attempts") and by the LangGraph harness (instead of "Sign in to your model provider again", which also told the desktop to ask for a sign-in). Tested on OpenAI's and xAI's real refusal bodies, captured, and Anthropic's as its API answers it. * Keep a gateway's own model and names, and wait for the first listing instead of answering empty A gateway that routes every vendor (OpenRouter, switchback) keeps the model its operator named while it lists it, and its models keep the names it gives them; one vendor's middle means nothing across all of them. OpenRouter's tilde aliases (~openai/...) carry their vendor's mark. A read that arrives while the boot listing is still on its way now waits for it, where it answered with nothing, hid the picker and ran the configured model. * Give the compose probe the harness's new imports The probe loads agent-langgraph-agui with stand-ins instead of its real packages. The picker work added three things that module now imports at load time: httpx (for the ChatGPT plan's model list), more names from tool_runtime (billing_refusal, provider_message, provider_refusal, current_tools), and a run's model and effort from current_tools(). CI's python3 has no httpx, so the two probe tests failed before a model was built. Add an httpx stand-in, the missing tool_runtime names, and a current_tools() that returns a run with no model or effort chosen, so the deployment's own configuration applies as before. * An operator's model is the default, and who may pick what depends on who uses OpenBot The operator's model wins. A model named in BOT_MODEL, or chosen on the Desktop setup screen, is the default for every run nobody picked a model for, on every provider, while the credential lists it. The middle of the provider's line applies only when nobody named one, or when the named one is not in the credential's list, which is how a ChatGPT plan given gpt-5.5 still lands on the plan's own middle model. To tell set from unset, nothing names a model on the operator's behalf any more: docker-compose.yml leaves the picked harness's BOT_MODEL empty when unset, and the Desktop no longer writes claude-sonnet-5-5 for an Anthropic key, whose setup screen asks for no model. One person on their own machine (OPENBOT_SINGLE_USER, the Desktop among them) is offered every model the credential lists, and nothing narrows it: no switch, no allowlist. In a team, an admin allows models in Admin, Providers, and people pick among those; until one is allowed people get the default and see no picker, so the old on/off switch is gone. The allowed-models dialog lists the same live models with their marks and thinking levels. An allowed model the provider stops listing drops out, and a pick saved on it runs on the default. * Keep the billing tests inside the default timeout, and leave the changelog to its own PR The two 429 tests waited out the AI SDK's real backoff (2 s, then 4 s), which CI's 5 s timeout cut short. The fake provider now answers with retry-after-ms: 0, the SDK's own seam for that wait, so every retry still happens and the tests assert all three attempts. The changelog for 0.3.1 belongs to its own PR. * Say what Only allowed models off means for a team: other models aren't refused, people still pick among the allowed * Mention the allowlist in Usage only where there is one One person on their own machine is offered every model and has no allowlist, but Admin > Providers' Usage still said it counted "runs the allowlist refused". Pass the page's team flag to the Usage section and say that only for a team.
1 parent 2ee1fad commit e240ea8

59 files changed

Lines changed: 14028 additions & 662 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎agent-claude-sdk/src/adapter.py‎

Lines changed: 71 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,21 +12,53 @@
1212
"""
1313

1414
import asyncio
15+
import logging
1516
from contextlib import suppress
1617
from dataclasses import dataclass, field
18+
from typing import get_args
1719

1820
from ag_ui.core import EventType, RunErrorEvent, RunFinishedEvent, RunStartedEvent
1921
from ag_ui_claude_sdk import ClaudeAgentAdapter
20-
from claude_agent_sdk import AssistantMessage, HookMatcher, ToolResultBlock, UserMessage
22+
from claude_agent_sdk import (
23+
AssistantMessage,
24+
EffortLevel,
25+
HookMatcher,
26+
ToolResultBlock,
27+
UserMessage,
28+
)
2129
from claude_agent_sdk.types import StreamEvent
2230

2331
from .replay import run_replayed_query, system_instructions
2432

33+
logger = logging.getLogger(__name__)
34+
35+
# The efforts `ClaudeAgentOptions.effort` takes (the CLI's `--effort`), read from the SDK itself.
36+
SDK_EFFORTS = frozenset(get_args(EffortLevel))
37+
2538

2639
class ModelAuthenticationError(Exception):
2740
pass
2841

2942

43+
def run_model(forwarded_props):
44+
"""The `(model, effort)` the server chose for this run, or `None` when it chose none.
45+
46+
The server sends `openbotModel: {"model": ..., "reasoningEffort": ...}` on both of this
47+
harness's paths and strips any a browser supplied, so it is the deployment's decision. A value
48+
of another shape is ignored. An effort the SDK does not take is not sent, and no effort means
49+
the model's own default.
50+
"""
51+
props = forwarded_props if isinstance(forwarded_props, dict) else {}
52+
choice = props.get("openbotModel")
53+
if not isinstance(choice, dict):
54+
return None
55+
model = choice.get("model")
56+
if not isinstance(model, str) or not model.strip():
57+
return None
58+
effort = choice.get("reasoningEffort")
59+
return model.strip(), effort if effort in SDK_EFFORTS else None
60+
61+
3062
@dataclass
3163
class _ClientBoundary:
3264
tool_ids: set[str]
@@ -47,6 +79,8 @@ class _Turn:
4779
# Tool calls Claude Code reported as failed, and the result the turn streamed for each.
4880
failed: set[str] = field(default_factory=set)
4981
failed_results: dict[str, str] = field(default_factory=dict)
82+
# The model the server chose when this query started, or that it was switched to since.
83+
model: str | None = None
5084

5185
def result(self, tool_id):
5286
if tool_id not in self.results:
@@ -73,13 +107,20 @@ def __init__(self, *args, model_only=False, **kwargs):
73107
self._segment_locks: dict[str, asyncio.Lock] = {}
74108

75109
def build_options(self, input_data=None, thread_id=None):
110+
# Read before the model endpoint drops its forwarded props: the server's choice of model
111+
# is the one prop it honours. Every query is built here, so a choice changed between runs
112+
# starts the next query on it.
113+
chosen = run_model(input_data.forwarded_props) if input_data else None
76114
if self._model_only and input_data:
77115
# The model endpoint accepts prompts and caller-owned tools, never SDK
78116
# configuration or an ungoverned state-management tool.
79117
input_data = input_data.model_copy(
80118
update={"state": None, "forwarded_props": {}}
81119
)
82120
options = super().build_options(input_data, thread_id)
121+
if chosen:
122+
# After the upstream merge, so the server's choice wins over a forwarded `model`.
123+
options.model, options.effort = chosen
83124
if not self._model_only and input_data:
84125
# The model endpoint gets its instructions in context. A full Bot gets its standing
85126
# role only as a system message, which a user prompt cannot carry.
@@ -313,6 +354,7 @@ async def run(self, input_data):
313354
message="This Claude query is waiting for its client tool result; no matching tool-use ID was supplied."
314355
)
315356
return
357+
await self._follow_model_choice(thread_id, turn, input_data)
316358
for tool_id, content in incoming.items():
317359
turn.result(tool_id).set_result(content)
318360
# Parallel client tools may return in separate HTTP requests.
@@ -329,7 +371,11 @@ async def run(self, input_data):
329371
message="The Claude query for this tool result is no longer running. Start a new message."
330372
)
331373
return
332-
turn = _Turn(tool_names={tool.name for tool in input_data.tools})
374+
chosen = run_model(input_data.forwarded_props)
375+
turn = _Turn(
376+
tool_names={tool.name for tool in input_data.tools},
377+
model=chosen[0] if chosen else None,
378+
)
333379
self._turns[thread_id] = turn
334380
turn.task = asyncio.create_task(self._pump(input_data, turn))
335381

@@ -361,6 +407,29 @@ async def run(self, input_data):
361407
if not released:
362408
await self.interrupt(thread_id)
363409

410+
async def _follow_model_choice(self, thread_id, turn, input_data):
411+
"""Move a running query to the model a tool result's run was sent with.
412+
413+
A tool result continues the SDK query that asked for it, so its options are the ones that
414+
query started with. A person can pick another model while a call waits on them, and the
415+
rest of that turn should answer on it: the SDK changes a running query's model
416+
(`ClaudeSDKClient.set_model`). It has no such call for effort, so a changed effort applies
417+
from the next query, which `build_options` starts on it.
418+
"""
419+
chosen = run_model(input_data.forwarded_props)
420+
if not chosen or chosen[0] == turn.model:
421+
return
422+
worker = (self._workers.get(thread_id) or {}).get("worker")
423+
client = getattr(worker, "_client", None)
424+
if client is None:
425+
return
426+
try:
427+
await client.set_model(chosen[0])
428+
except Exception as error: # The turn finishes on the model it has rather than failing.
429+
logger.warning("The running Claude query kept its model: %s", type(error).__name__)
430+
return
431+
turn.model = chosen[0]
432+
364433
async def interrupt(self, thread_id=None):
365434
threads = [thread_id] if thread_id else list(self._turns)
366435
for current in threads:
Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
"""The server's choice of model and effort reaches the Claude Code process, on both paths.
2+
3+
The server sends `forwardedProps.openbotModel` on every run: to `/model` for its built-in Bots on a
4+
Claude plan, and to `/` when this harness is the Bot a person chats with. What the choice must
5+
reach is the command the SDK starts Claude Code with (`--model`, `--effort`), so that is what is
6+
checked, built by the SDK's own transport from the options this adapter makes. Queries run against
7+
the controlled SDK client in `test_tool_continuation`, which runs the adapter's real hooks.
8+
"""
9+
10+
import asyncio
11+
import sys
12+
from pathlib import Path
13+
14+
import pytest
15+
from ag_ui.core import RunAgentInput
16+
from claude_agent_sdk._internal.transport.subprocess_cli import SubprocessCLITransport
17+
18+
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
19+
from src.adapter import OpenBotClaudeAgentAdapter
20+
from test_tool_continuation import events, request, sdk, tool_name # noqa: F401 (fixtures)
21+
22+
23+
def run_input(forwarded_props):
24+
return RunAgentInput.model_validate(
25+
{
26+
"threadId": "thread",
27+
"runId": "run",
28+
"state": None,
29+
"messages": [{"id": "user", "role": "user", "content": "Plan my week"}],
30+
"tools": [],
31+
"context": [],
32+
"forwardedProps": forwarded_props,
33+
}
34+
)
35+
36+
37+
def command(options):
38+
"""The Claude Code command line the SDK builds from these options."""
39+
options.cli_path = "/not-executed/claude"
40+
return SubprocessCLITransport(prompt="contract only", options=options)._build_command()
41+
42+
43+
def flag(cmd, name):
44+
return cmd[cmd.index(name) + 1] if name in cmd else None
45+
46+
47+
@pytest.mark.parametrize("model_only", [True, False])
48+
@pytest.mark.parametrize(
49+
("choice", "model", "effort"),
50+
[
51+
({"model": "claude-opus-5-5", "reasoningEffort": "xhigh"}, "claude-opus-5-5", "xhigh"),
52+
({"model": "claude-sonnet-5", "reasoningEffort": "low"}, "claude-sonnet-5", "low"),
53+
({"model": "claude-haiku-4-5"}, "claude-haiku-4-5", None),
54+
# Efforts Claude Code does not take are not sent: the model's own default applies.
55+
({"model": "claude-haiku-4-5", "reasoningEffort": "minimal"}, "claude-haiku-4-5", None),
56+
({"model": "claude-haiku-4-5", "reasoningEffort": "none"}, "claude-haiku-4-5", None),
57+
],
58+
)
59+
def test_the_servers_choice_reaches_claude_code(model_only, choice, model, effort):
60+
adapter = OpenBotClaudeAgentAdapter(name="choice", model_only=model_only)
61+
options = adapter.build_options(run_input({"openbotModel": choice}), "thread")
62+
63+
assert (options.model, options.effort) == (model, effort)
64+
cmd = command(options)
65+
assert flag(cmd, "--model") == model
66+
assert flag(cmd, "--effort") == effort
67+
asyncio.run(adapter.shutdown())
68+
69+
70+
@pytest.mark.parametrize("model_only", [True, False])
71+
@pytest.mark.parametrize(
72+
"props", [{}, {"openbotModel": {"model": 5}}, {"openbotModel": "claude-opus-5-5"}]
73+
)
74+
def test_without_a_choice_claude_code_keeps_its_own_model(model_only, props):
75+
adapter = OpenBotClaudeAgentAdapter(name="choice", model_only=model_only)
76+
cmd = command(adapter.build_options(run_input(props), "thread"))
77+
78+
assert "--model" not in cmd
79+
assert "--effort" not in cmd
80+
asyncio.run(adapter.shutdown())
81+
82+
83+
def test_the_servers_choice_wins_over_a_forwarded_model():
84+
adapter = OpenBotClaudeAgentAdapter(name="choice")
85+
options = adapter.build_options(
86+
run_input({"model": "claude-haiku-4-5", "openbotModel": {"model": "claude-opus-5-5"}}),
87+
"thread",
88+
)
89+
90+
assert options.model == "claude-opus-5-5"
91+
asyncio.run(adapter.shutdown())
92+
93+
94+
@pytest.mark.parametrize("model_only", [True, False])
95+
def test_a_choice_changed_between_runs_starts_the_next_query_on_it(sdk, model_only): # noqa: F811
96+
async def scenario():
97+
adapter = OpenBotClaudeAgentAdapter(name="choice", model_only=model_only)
98+
try:
99+
first = request(run="run-1", tool_count=0)
100+
first.forwarded_props = {"openbotModel": {"model": "claude-sonnet-5", "reasoningEffort": "high"}}
101+
assert (await events(adapter, first))[-1].type == "RUN_FINISHED"
102+
second = request(run="run-2", tool_count=0)
103+
second.forwarded_props = {"openbotModel": {"model": "claude-opus-5-5", "reasoningEffort": "low"}}
104+
assert (await events(adapter, second))[-1].type == "RUN_FINISHED"
105+
third = request(run="run-3", tool_count=0)
106+
third.forwarded_props = {"openbotModel": {"model": "claude-opus-5-5"}}
107+
assert (await events(adapter, third))[-1].type == "RUN_FINISHED"
108+
109+
assert [(client.options.model, client.options.effort) for client in sdk] == [
110+
("claude-sonnet-5", "high"),
111+
("claude-opus-5-5", "low"),
112+
("claude-opus-5-5", None),
113+
]
114+
finally:
115+
await adapter.shutdown()
116+
117+
asyncio.run(scenario())
118+
119+
120+
@pytest.mark.parametrize("model_only", [True, False])
121+
def test_a_tool_result_sent_with_another_model_moves_the_running_query_to_it(sdk, model_only): # noqa: F811
122+
async def scenario():
123+
adapter = OpenBotClaudeAgentAdapter(name="choice", model_only=model_only)
124+
try:
125+
first = request(run="run-1")
126+
first.forwarded_props = {"openbotModel": {"model": "claude-sonnet-5"}}
127+
assert any(event.type == "TOOL_CALL_END" for event in await events(adapter, first))
128+
129+
same = request(run="run-2", results=[("thread-a-tool-0", "page title")])
130+
same.forwarded_props = {"openbotModel": {"model": "claude-opus-5-5"}}
131+
resumed = await events(adapter, same)
132+
133+
assert resumed[-1].type == "RUN_FINISHED"
134+
assert len(sdk) == 1, "the tool result continues the query that asked for it"
135+
assert sdk[0].models_set == ["claude-opus-5-5"]
136+
assert sdk[0].results["thread-a-tool-0"] == [{"type": "text", "text": "page title"}]
137+
finally:
138+
await adapter.shutdown()
139+
140+
asyncio.run(scenario())
141+
142+
143+
@pytest.mark.parametrize("model_only", [True, False])
144+
def test_a_tool_result_on_the_same_model_leaves_the_query_alone(sdk, model_only): # noqa: F811
145+
async def scenario():
146+
adapter = OpenBotClaudeAgentAdapter(name="choice", model_only=model_only)
147+
try:
148+
first = request(run="run-1")
149+
first.forwarded_props = {"openbotModel": {"model": "claude-sonnet-5"}}
150+
await events(adapter, first)
151+
same = request(run="run-2", results=[("thread-a-tool-0", "page title")])
152+
same.forwarded_props = {"openbotModel": {"model": "claude-sonnet-5"}}
153+
assert (await events(adapter, same))[-1].type == "RUN_FINISHED"
154+
155+
assert sdk[0].models_set == []
156+
finally:
157+
await adapter.shutdown()
158+
159+
asyncio.run(scenario())

‎agent-claude-sdk/tests/test_tool_continuation.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,7 @@ def __init__(self, options):
7373
self.queries = []
7474
self.results = {}
7575
self.interrupted = False
76+
self.models_set = []
7677
self.disconnected = False
7778
self.disconnected_event = asyncio.Event()
7879
self.hooks_entered = asyncio.Event()
@@ -88,6 +89,10 @@ async def disconnect(self):
8889
async def interrupt(self):
8990
self.interrupted = True
9091

92+
async def set_model(self, model=None):
93+
# `ClaudeSDKClient.set_model`: the running query's next request uses `model`.
94+
self.models_set.append(model)
95+
9196
async def query(self, prompt, session_id):
9297
if not isinstance(prompt, str):
9398
self.input_messages = [message async for message in prompt]

0 commit comments

Comments
 (0)