Skip to content

Commit ca05b73

Browse files
easonLiangWorldedtecheasonliang28edelaunaeasonLiangWorldedtechdsh-agent
authored
feat(api): abort signal support for anthropic, anthropic-vertex, xai, minimax (#1293)
* feat(api): abort signal and timeout support for anthropic, anthropic-vertex, xai, minimax - completePrompt: forward CompletePromptOptions abortSignal/timeoutMs into the SDK request for AnthropicHandler, AnthropicVertexHandler, XAIHandler, and MiniMaxHandler (request options built only when a signal/timeout is provided, preserving existing behavior) - createMessage: bridge metadata?.abortSignal into a per-request AbortController using the Bedrock pattern (pre-aborted guard + { once: true } listener) and pass the internal signal as the SDK request signal; existing client-level timeout mechanisms are untouched - xai: rethrow AbortError unmodified from createMessage/completePrompt so callers can detect error.name === 'AbortError' - tests: port reference completePrompt signal/timeout propagation tests and add per-provider createMessage bridging tests (pre-aborted signal rejects with AbortError; mid-flight external abort cancels the request) * ci: re-run e2e-mock on flaky restart-persistence (uses openrouter; this PR only touches anthropic family) * fix(api): address CodeRabbit review on anthropic family abort handling - anthropic.ts: use options?.timeoutMs !== undefined (was truthy) so a caller-supplied timeoutMs: 0 is forwarded to the SDK instead of silently dropped; all 4 family providers now share the same defined-check - xai.ts: rethrow the OpenAI SDK's APIUserAbortError (exported from openai v5) unmodified from createMessage/completePrompt alongside native AbortError, since the SDK throws it when the request signal aborts and it would otherwise be mangled by handleOpenAIError - tests: anthropic.spec.ts regression test asserting timeoutMs: 0 reaches the SDK as timeout: 0; xai.spec.ts exposes the real APIUserAbortError in the openai mock and asserts the SDK abort error surfaces as the same instance (unwrapped) through both createMessage and completePrompt * fix(api): type xAI streaming request body with the OpenAI SDK type - xai.ts: declare the streaming request body as OpenAI.Responses.ResponseCreateParamsStreaming instead of Record<string, any>, so the full request shape (incl. include/reasoning) is typechecked against the SDK and the as any on the create() call is no longer needed - xai.ts: type the stream as AsyncIterable<OpenAI.Responses.ResponseStreamEvent> (matching the codebase pattern in mimo.ts/openai.ts) and drop the as unknown as AsyncIterable<any> double cast, since the SDK create() streaming overload already returns an AsyncIterable stream - eslint-suppressions.json: reduce @typescript-eslint/no-explicit-any count for api/providers/xai.ts from 7 to 3 (four any usages removed) * fix(api): detach external abort listeners and cover timeoutMs 0 - anthropic-vertex and xai createMessage retain the external abort listener callback and remove it in a finally block, so a long-lived caller signal no longer accumulates one listener (and closed-over controller) per completed request; specs assert the same callback is detached after stream completion. - anthropic-vertex completePrompt spec adds the timeoutMs: 0 case asserting { timeout: 0 } reaches messages.create, guarding the defined-value check against a truthiness regression. * refactor(api): map xai tool_choice to Responses API shape instead of casting * fix(api): flatten xai allowed_tools entries for the Responses API mapToolChoice forwarded Chat Completions allowlist entries ({ type: "function", function: { name } }) to responses.create, but the Responses API expects { type: "function", name }. Flatten function-shaped entries and leave other entry types untouched; add a regression test. * test(api): cover abort-bridge and request-body mutation gaps Add focused tests that kill the surviving and uncovered changed-code mutants reported by the mutation-diff gate on this branch: - xai: custom tool_choice mapping, allowed_tools passthrough for unmappable entries, default-model max_output_tokens/temperature, omitted tool parameters without tools, native AbortError passthrough in completePrompt, exact Responses API request body, telemetry capture for createMessage failures, and the once option on the external abort listener. - anthropic: a pending external signal does not pre-abort the bridged signal, the external abort listener is registered with the once option, and the default-model beta header is comma-joined. - anthropic-vertex: message_start usage defaults outputTokens to zero when output_tokens is omitted. - minimax: completePrompt sends the configured temperature, user message and stream flag; the abort bridge does not pre-abort pending external signals and registers its listener with the once option. Local mutation-diff gate: 7 surviving and 1 uncovered changed-code mutants remain; all are unkillable without source changes (dead inner default in anthropic.ts, spread-overridden stream flag and always-true guards in xai.ts, single-element beta join and inert case label in anthropic-vertex.ts). * fix(api): remove dead code flagged by the mutation gate in anthropic, xai, and anthropic-vertex - anthropic.ts: collapse the inner prompt-caching switch, whose 19 case branches all executed an identical body plus an unreachable default, into that body: push the prompt-caching beta, build the anthropic-beta header, and attach the abort signal once before messages.create. Pinned by anthropic.spec.ts (comma-joined beta header and pending-abort-signal assertions). Deviation: tsconfig noImplicitReturns forbids a string switch without a default, so the identical branches were collapsed instead of deleting the default alone. - xai.ts: drop the call-site `stream: true` spread override from responses.create (the ResponseCreateParamsStreaming type requires the flag in the request body, so the body literal keeps it and is what is sent); collapse the model.maxTokens and model.temperature guards into unconditional assignments; bind metadata?.tool_choice and metadata?.parallelToolCalls once at the top of createMessage instead of re-reading the metadata at the use sites. Pinned by xai.spec.ts exact body assertions (stream: true, max_output_tokens 65_536, temperature 0, tool_choice "auto", parallel_tool_calls true). - anthropic-vertex.ts: send the single beta label directly (betas[0]) instead of building an array and joining it, and delete the no-op content_block_stop case. Pinned by anthropic-vertex.spec.ts exact anthropic-beta header assertions. - xai.spec.ts: add a createMessage test for parallelToolCalls: false so the defaulting of parallel_tool_calls stays pinned to an exact value. * fix(api): attach anthropic abort signal via spread to drop the surviving guard mutant * fix(api): detach external abort listeners when anthropic/minimax requests end * test: assert non-cached Anthropic path rejects with AbortError on external abort * Address review: shared abort bridge, canonical xAI aborts, beta join, cleanup coverage - Extract the four copies of the external-abort bridge (anthropic, anthropic-vertex, minimax, xai) into bridgeExternalAbortSignal in utils/abort-signal.ts (single canonical comment, unit-tested incl. a negative control that dispose detaches the exact retained callback). - xai: normalize cancellations through isRequestAborted + createAbortError in both createMessage and completePrompt; the raw OpenAI SDK APIUserAbortError (name "Error", message "Request was aborted.") no longer escapes to callers' AbortError guards. - anthropic-vertex: send anthropic-beta as betas.join(",") again so a second beta added to the builder later is not silently dropped. - Move the xAI tool-choice mapping (mapToolChoice + typed request body + unconditional max_output_tokens/temperature) and the vertex content_block_stop removal out of this abort-wiring unit per review: the hunks return to a separate PR; xai.ts keeps its main-side lint suppression count (7) accordingly. - Pin listener cleanup beyond normal completion: vertex and xai specs now cover create-rejects and consumer-early-return paths (vertex uses an inline finally, so stream-cleanup coverage does not reach it). Verification at this head: 273/273 green across the six touched specs; tsc --noEmit 0 errors; eslint --max-warnings=0 clean on touched files; prettier clean. * chore(api): remove Stryker suppression comments from touched files Per repo policy (AGENTS.md: Stryker suppression comments are banned; the mutation gate rejects them in touched files). Equivalence arguments for the resulting mutants are recorded in the PR body. * fix(api): normalize stream-iteration aborts in the xAI responses path CodeRabbit: an abort landing after responses.create resolved escaped as the OpenAI SDK's APIUserAbortError (name "Error") from the yield* over processResponsesApiStream, outside the create catch, so callers classified the cancellation as a provider failure. Wrap the stream iteration and normalize through isRequestAborted/createAbortError to the series contract (name AbortError, "The xAI request was aborted"). Regression test pins the path; negative control: reverting the wrap fails exactly that test (1 red / 37 green). * fix(api): reject non-finite and cap overflowing mergeAbortSignalAndTimeout delays Port of the #1300 timeout guard finding (CodeRabbit): Node's AbortSignal.timeout throws ERR_OUT_OF_RANGE for non-integer delays (Infinity) and clamps delays above 2^31-1 ms to 1 ms, so a non-finite timeoutMs threw a RangeError instead of running the request, and an overflowing one aborted almost immediately. The helper now treats non-finite values as "no timeout" and caps finite delays at 2^31-1 ms. Regression tests added; negative controls (reverting the guard) fail exactly the two new tests. * fix(api): round fractional timeouts before AbortSignal.timeout CodeRabbit follow-up on the timeout guard: Node validates AbortSignal.timeout delays as uint32, so finite fractions (1500.5, 0.5) threw ERR_OUT_OF_RANGE just like Infinity did. The helper now rounds delays up (a positive fraction never becomes 0 ms) before capping at 2^31-1 ms. Regression test mergeAbortSignalAndTimeout (undefined, 1.5); negative control (reverting the rounding) fails exactly that test. * test(api): assert the rounded delay reaching AbortSignal.timeout CodeRabbit: the fractional-timeout test only proved the call did not throw and eventually aborted - a round-down mutation (1.5 -> 1) would still pass. The test now spies AbortSignal.timeout and pins the received delay: 2 for 1.5 and 1 for 0.5 (never 0). Negative control: mutating Math.ceil to Math.floor fails exactly this test. * fix(anthropic-family): forward timeoutMs only when the SDK can take it The four completePrompt copies forwarded any defined timeoutMs. The Anthropic SDK turns timeout: 0 into an immediate abort and its request validation throws on negative, NaN and Infinity, so the series convention (timeoutMs <= 0 or non-finite means no explicit timeout) must leave the option absent. Fractional delays are rounded up, matching mergeAbortSignalAndTimeout. hasUsableTimeout is the shared predicate: it also backs mergeAbortSignalAndTimeout, so the rule is single-sourced for both the option and the signal. * style: apply repo Prettier to the merge resolution * refactor(api): share one SDK timeout conversion across the anthropic family The four SDK option paths rounded fractional timeouts up but did not cap them, so an overflowing timeout would be clamped to 1 ms by Node. Add toSdkTimeoutMs (+ MAX_TIMEOUT_MS) next to hasUsableTimeout, use it in mergeAbortSignalAndTimeout and at all four provider sites, and cover the fractional/cap boundaries. * style: apply repo Prettier to the timeout helper import * test(providers): pin timeout rounding and the timer cap across the anthropic family Review follow-ups: move the mergeAbortSignalAndTimeout JSDoc back above its function (MAX_TIMEOUT_MS and toSdkTimeoutMs now sit before it); the completePrompt timeout tests in the anthropic, anthropic-vertex, minimax and xai specs are now it.each cases covering a whole-number, a fractional (12.5 -> 13) and an over-cap (MAX+1 -> MAX) value, so forwarding the raw timeoutMs instead of calling toSdkTimeoutMs fails in every provider. Affected suites 364 passed. * fix(api): lint request timeouts and spec loops, fix existing violations --------- Co-authored-by: Eason Liang <easonliang28@gmail.com> Co-authored-by: edelauna <54631123+edelauna@users.noreply.github.com> Co-authored-by: easonLiangWorldedtech <eason.liang@worldedtech.com> Co-authored-by: dsh-agent <dsh@local> Co-authored-by: Elliott de Launay <edelauna@gmail.com>
1 parent 8383de4 commit ca05b73

25 files changed

Lines changed: 2240 additions & 337 deletions

‎packages/config-eslint/api-provider-hygiene.js‎

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,17 @@
33
// Paths are relative to the `src` package, where this config is loaded.
44

55
const sharedAbortHelpers =
6-
"mergeAbortSignalAndTimeout|mergeAbortSignals|throwIfAborted|isRequestAborted|createAbortError|rejectOnAbort|resolveModelWithAbort"
6+
"mergeAbortSignalAndTimeout|mergeAbortSignals|throwIfAborted|isRequestAborted|createAbortError|rejectOnAbort|resolveModelWithAbort|bridgeExternalAbortSignal|hasUsableTimeout|toSdkTimeoutMs"
77

88
// Names that usually hold a secret or an endpoint that can embed credentials.
99
// `cacheKey` and `contextTokens` must not match.
1010
const sensitiveName = "^(url|apiKey|secret|password)$|(Url|URL|ApiKey|Secret|Token)$"
1111

12+
// A per-request `timeoutMs` from CompletePromptOptions. `this.timeoutMs` is the client-level
13+
// timeout from getApiRequestTimeout, which is already valid, so it must not match.
14+
const rawRequestTimeout =
15+
":matches(Identifier[name='timeoutMs'], MemberExpression[property.name='timeoutMs']:not([object.type='ThisExpression']), ChainExpression[expression.property.name='timeoutMs'])"
16+
1217
export const apiProviderHygieneSelectors = [
1318
{
1419
selector: `:matches(FunctionDeclaration, VariableDeclarator)[id.name=/^(${sharedAbortHelpers})$/]`,
@@ -20,6 +25,16 @@ export const apiProviderHygieneSelectors = [
2025
message:
2126
"Use mergeAbortSignalAndTimeout for a variable delay. AbortSignal.timeout throws on Infinity and fractions, and clamps large values to 1 ms.",
2227
},
28+
{
29+
selector: `:matches(Property[key.name='timeout'], AssignmentExpression[left.property.name='timeout']) > ${rawRequestTimeout}`,
30+
message:
31+
"Do not forward a request timeoutMs as is. A SDK treats 0 as an immediate abort and throws on negative, non-finite or fractional values. Use hasUsableTimeout and toSdkTimeoutMs.",
32+
},
33+
{
34+
selector: "Literal[value=2147483647]",
35+
message:
36+
"Do not write the maximum timer delay by hand. Use MAX_TIMEOUT_MS or toSdkTimeoutMs from providers/utils/abort-signal.",
37+
},
2338
{
2439
selector: `CallExpression[callee.object.name='console'] TemplateLiteral > :matches(Identifier[name=/${sensitiveName}/], MemberExpression[property.name=/${sensitiveName}/])`,
2540
message:
@@ -35,3 +50,12 @@ export const apiProviderHygieneConfig = {
3550
"no-restricted-syntax": ["error", ...apiProviderHygieneSelectors],
3651
},
3752
}
53+
54+
// Spec files under src/api. A loop of expects in one test stops at the first failing case and hides the rest.
55+
export const apiProviderTestSelectors = [
56+
{
57+
selector: "ForOfStatement[right.type='ArrayExpression']:has(CallExpression[callee.name='expect'])",
58+
message:
59+
"Use it.each for repeated cases. A loop of expects in one test stops at the first failing case and does not name it.",
60+
},
61+
]

‎packages/config-eslint/api-provider-hygiene.test.js‎

Lines changed: 87 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ import { describe, it } from "node:test"
44
import { Linter } from "eslint"
55
import typescriptEslint from "typescript-eslint"
66

7-
import { apiProviderHygieneConfig } from "./api-provider-hygiene.js"
7+
import { apiProviderHygieneConfig, apiProviderTestSelectors } from "./api-provider-hygiene.js"
88

99
const linter = new Linter({ configType: "flat" })
1010
const config = [{ files: ["**/*.ts"], languageOptions: { parser: typescriptEslint.parser } }, apiProviderHygieneConfig]
@@ -88,3 +88,89 @@ describe("api provider hygiene: sensitive values in logs", () => {
8888
})
8989
}
9090
})
91+
92+
describe("api provider hygiene: request timeout forwarding", () => {
93+
for (const code of [
94+
"create(body, { timeout: options.timeoutMs })",
95+
"create(body, { timeout: options?.timeoutMs })",
96+
"create(body, { signal, timeout: timeoutMs })",
97+
"requestOptions.timeout = options.timeoutMs",
98+
"requestOptions.timeout = timeoutMs",
99+
]) {
100+
it(`flags: ${code}`, () => {
101+
assert.equal(lint(code).length, 1)
102+
})
103+
}
104+
105+
for (const code of [
106+
"new OpenAI({ timeout: this.timeoutMs })",
107+
"requestOptions.timeout = toSdkTimeoutMs(timeoutMs)",
108+
"create(body, { timeout: toSdkTimeoutMs(options.timeoutMs) })",
109+
"if (hasUsableTimeout(timeoutMs)) requestOptions.timeout = toSdkTimeoutMs(timeoutMs)",
110+
"const timeoutMs = options?.timeoutMs",
111+
"mergeAbortSignalAndTimeout(signal, options?.timeoutMs)",
112+
]) {
113+
it(`allows: ${code}`, () => {
114+
assert.deepEqual(lint(code), [])
115+
})
116+
}
117+
})
118+
119+
describe("api provider hygiene: timeout helpers", () => {
120+
for (const code of [
121+
"const MAX = 2147483647",
122+
"Math.min(Math.ceil(timeoutMs), 2_147_483_647)",
123+
"function hasUsableTimeout(t?: number) { return true }",
124+
"const toSdkTimeoutMs = (t: number) => t",
125+
"function bridgeExternalAbortSignal() {}",
126+
]) {
127+
it(`flags: ${code}`, () => {
128+
assert.equal(lint(code).length, 1)
129+
})
130+
}
131+
132+
it("allows importing the timeout helpers", () => {
133+
assert.deepEqual(
134+
lint('import { hasUsableTimeout, toSdkTimeoutMs } from "./utils/abort-signal"\ntoSdkTimeoutMs(1)'),
135+
[],
136+
)
137+
})
138+
139+
it("allows the literal in the helper module", () => {
140+
assert.deepEqual(lint("export const MAX_TIMEOUT_MS = 2_147_483_647", "api/providers/utils/abort-signal.ts"), [])
141+
})
142+
})
143+
144+
describe("api provider hygiene: spec loops", () => {
145+
const testLinter = new Linter({ configType: "flat" })
146+
const testConfig = [
147+
{ files: ["**/*.ts"], languageOptions: { parser: typescriptEslint.parser } },
148+
{
149+
files: ["api/**/__tests__/**/*.spec.ts"],
150+
rules: { "no-restricted-syntax": ["error", ...apiProviderTestSelectors] },
151+
},
152+
]
153+
const lintSpec = (code) =>
154+
testLinter
155+
.verify(code, testConfig, { filename: "api/providers/__tests__/example.spec.ts" })
156+
.map((m) => m.message)
157+
158+
for (const code of [
159+
"it('x', () => { for (const v of [0, -5]) { expect(f(v)).toBe(1) } })",
160+
"it('x', async () => { for (const v of [0, -5]) { await run(v); expect(f(v)).toBe(1) } })",
161+
]) {
162+
it(`flags a loop of expects over a literal array: ${code}`, () => {
163+
assert.equal(lintSpec(code).length, 1)
164+
})
165+
}
166+
167+
for (const code of [
168+
"it.each([0, -5])('x %s', (v) => { expect(f(v)).toBe(1) })",
169+
"it('x', () => { for (const v of [0, -5]) { run(v) } })",
170+
"it('x', () => { for (const v of cases) { expect(f(v)).toBe(1) } })",
171+
]) {
172+
it(`allows: ${code}`, () => {
173+
assert.deepEqual(lintSpec(code), [])
174+
})
175+
}
176+
})

0 commit comments

Comments
 (0)