Skip to content

Commit ab1b14d

Browse files
zkasuranclaude
andcommitted
fix(server-utils): Match google-genai per-message config replacement on chat spans
@google/genai resolves each request's config as `params.config ?? chat.config`, so a per-message config given to chat.sendMessage() or chat.sendMessageStream() replaces the create-time config wholesale rather than merging with it. The previous shallow merge welded create-time fields such as systemInstruction, tools and sampling settings onto message spans that did not actually send them, over-reporting what the request carried. Fall back to the create-time config only when the message carries no config of its own. Lock the corrected attributes in the per-message config test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent e0d7e86 commit ab1b14d

2 files changed

Lines changed: 20 additions & 10 deletions

File tree

‎packages/server-utils/src/ai/google-genai/index.ts‎

Lines changed: 12 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -257,9 +257,10 @@ function asConfigObject(config: unknown): Record<string, unknown> | undefined {
257257
/**
258258
* Merge the parameters captured at `chats.create()` time onto a `chat.sendMessage()` /
259259
* `chat.sendMessageStream()` call, so the config that is defined once on the chat lands on every
260-
* message span. The create-time config is the default and a per-message config overrides it key by
261-
* key. Only `model` and `config` are inherited; the create `history` is intentionally left off the
262-
* message spans. See issue #20086.
260+
* message span. The SDK resolves the request config as `params.config ?? chat.config`, so a
261+
* per-message config replaces the create-time config wholesale and the create-time config only
262+
* applies when the message omits one. Only `model` and `config` are inherited; the create `history`
263+
* is intentionally left off the message spans. See issue #20086.
263264
*/
264265
function mergeChatCreateParams(
265266
chatCreateParams: Record<string, unknown> | undefined,
@@ -275,10 +276,15 @@ function mergeChatCreateParams(
275276
merged.model = chatCreateParams.model;
276277
}
277278

278-
const createConfig = asConfigObject(chatCreateParams.config);
279+
// @google/genai sends `params.config ?? chat.config`, so a per-message config replaces the
280+
// create-time config wholesale rather than merging into it. Fall back to the create-time config
281+
// only when the message did not carry one, otherwise the span reports fields that were not sent.
279282
const callConfig = asConfigObject(callParams?.config);
280-
if (createConfig || callConfig) {
281-
merged.config = { ...createConfig, ...callConfig };
283+
if (!callConfig) {
284+
const createConfig = asConfigObject(chatCreateParams.config);
285+
if (createConfig) {
286+
merged.config = createConfig;
287+
}
282288
}
283289

284290
return merged;

‎packages/server-utils/test/ai/lib/tracing/google-genai.test.ts‎

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -139,7 +139,7 @@ describe('instrumentGoogleGenAIClient chat config propagation (#20086)', () => {
139139
expect(data[GEN_AI_SYSTEM_INSTRUCTIONS]).toBe('[{"type":"text","content":"You are a friendly robot."}]');
140140
});
141141

142-
it('lets a per-message config override the config from chats.create()', async () => {
142+
it('replaces the chats.create() config when a message provides its own', async () => {
143143
const endedSpans = setupClient();
144144
const instrumented = instrumentGoogleGenAIClient(createFakeClient());
145145

@@ -149,10 +149,14 @@ describe('instrumentGoogleGenAIClient chat config propagation (#20086)', () => {
149149
await chat.sendMessage({ message: 'Tell me a joke', config: { temperature: 0.1 } });
150150

151151
const data = spanToJSON(endedSpans[0]!).data;
152-
// Per-call temperature wins; the untouched create-time values still come through.
152+
// @google/genai resolves the request config as `params.config ?? chat.config`, so the
153+
// per-message config is sent on its own. The create-time fields it omits are not part of the
154+
// request, so they must not appear on the span.
153155
expect(data[GEN_AI_REQUEST_TEMPERATURE]).toBe(0.1);
154-
expect(data[GEN_AI_REQUEST_TOP_P]).toBe(0.9);
155-
expect(data[GEN_AI_REQUEST_MAX_TOKENS]).toBe(150);
156+
expect(data[GEN_AI_REQUEST_TOP_P]).toBeUndefined();
157+
expect(data[GEN_AI_REQUEST_MAX_TOKENS]).toBeUndefined();
158+
expect(data[GEN_AI_TOOL_DEFINITIONS]).toBeUndefined();
159+
expect(data[GEN_AI_SYSTEM_INSTRUCTIONS]).toBeUndefined();
156160
});
157161

158162
it('does not leak chat config onto models.generateContent spans', async () => {

0 commit comments

Comments
 (0)