Skip to content

Commit 1f3cdf2

Browse files
committed
test(api): cover the accepted prompt cache TTL on the gateway call site
What: add a dedicated `anthropic/claude-3.5-sonnet` catalog row carrying `promptCacheTtlSupport: ["1h"]`, and a test asserting the third argument to `addCacheBreakpoints` resolves to `"1h"` when the knob is set. Why: the existing pair only proved the reject path (`lastCall[2]` undefined) and a bare `toHaveBeenCalled`. Nothing proved the accept path, so a regression that dropped the third argument at the call site stayed invisible. The real gateway catalog never sets `promptCacheTtlSupport`, so the capability has to be injected by the test catalog rather than assumed from a real model. Impact: test-only. The shared haiku row keeps no TTL capability, so the existing reject-path test still asserts fail-closed behavior.
1 parent 1f98244 commit 1f3cdf2

1 file changed

Lines changed: 34 additions & 0 deletions

File tree

‎src/api/providers/__tests__/vercel-ai-gateway.spec.ts‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,22 @@ vitest.mock("../fetchers/modelCache", () => ({
9898
cacheReadsPrice: 0.1,
9999
description: "Claude 3.5 Haiku",
100100
},
101+
// The real gateway catalog never sets `promptCacheTtlSupport`, so this row
102+
// exists only to inject the capability the accept path needs. It is kept
103+
// separate from the shared haiku row so the reject-path test keeps a model
104+
// with no TTL capability.
105+
"anthropic/claude-3.5-sonnet": {
106+
maxTokens: 8192,
107+
contextWindow: 200000,
108+
supportsImages: true,
109+
supportsPromptCache: true,
110+
promptCacheTtlSupport: ["1h"],
111+
inputPrice: 3,
112+
outputPrice: 15,
113+
cacheWritesPrice: 3.75,
114+
cacheReadsPrice: 0.3,
115+
description: "Claude 3.5 Sonnet",
116+
},
101117
"openai/gpt-4o": {
102118
maxTokens: 16000,
103119
contextWindow: 128000,
@@ -472,6 +488,24 @@ describe("VercelAiGatewayHandler", () => {
472488
expect(lastCall![2]).toBeUndefined()
473489
})
474490

491+
it("passes the resolved ttl for a gateway model with 1h TTL capability", async () => {
492+
// The runtime catalog never sets `promptCacheTtlSupport`, so the accept path is
493+
// only reachable with an injected capability. The dedicated catalog row keeps
494+
// the reject-path model capability-free.
495+
const { addCacheBreakpoints } = await import("../../transform/caching/vercel-ai-gateway")
496+
const handler = new VercelAiGatewayHandler(
497+
makeApiHandlerOptions({
498+
...mockOptions,
499+
vercelAiGatewayModelId: "anthropic/claude-3.5-sonnet",
500+
promptCacheTtl: "1h",
501+
}),
502+
)
503+
504+
await handler.createMessage("You are a helpful assistant.", [{ role: "user", content: "Hello" }]).next()
505+
506+
expect(vitest.mocked(addCacheBreakpoints).mock.calls.at(-1)![2]).toBe("1h")
507+
})
508+
475509
it("sets correct max_completion_tokens", async () => {
476510
const handler = new VercelAiGatewayHandler(mockOptions)
477511

0 commit comments

Comments
 (0)