Add --extended-ttl-cache to `thv llm setup` to opt clients into the one-hour prompt cache lifetime

[aponcedeleonch]

User story

As a developer running AI coding tools through a ToolHive-configured LLM gateway, I want a single thv llm setup flag that opts my tools into the one-hour prompt-cache lifetime, so that my cached context survives normal breaks in work instead of expiring after five minutes and re-billing the whole conversation on the next turn.

Scope

Prompt-cache lifetime is decided by the client, not by the gateway. There is no server-side setting for it. When a client reaches Claude through a custom base URL, which is exactly what thv llm setup configures, it falls into the billing bucket whose default cache lifetime is five minutes rather than one hour. The result is that a cache expires during any ordinary pause in work, and the next turn reprocesses the full conversation at uncached rates.

This story adds a single global flag, --extended-ttl-cache, to thv llm setup and thv llm config set. It asks every configured client that has a prompt-cache-lifetime control to use the one-hour lifetime.

Only some clients have such a control. Of the seven clients thv llm setup configures, Claude Code is currently the only one that exposes one; Claude Desktop, Codex, Gemini CLI, Cursor, VS Code, and Xcode do not. The flag is deliberately global rather than Claude-Code-specific so it keeps working unchanged as other clients gain support, but its promise is honest about reach: it applies the setting wherever a control exists and says plainly where it did not.

The one-hour lifetime is not free. Cache writes bill at twice the base input rate instead of 1.25 times, and a one-hour entry needs three requests to break even where a five-minute entry needs two. It pays off for work that idles between five and sixty minutes and costs more for continuous bursts that never idle past five minutes. That tradeoff is why the flag is opt-in rather than the default.

Acceptance criteria

1. Running thv llm setup --extended-ttl-cache configures every client it would otherwise configure, and additionally requests the one-hour prompt-cache lifetime for each of those clients that supports one.

2. For Claude Code, the one-hour lifetime covers both of its request buckets: the main conversation, and the separate bucket holding subagents, workflows, forks, compaction, and session titles. Neither bucket is left at the five-minute default.

3. A developer whose installed Claude Code predates the per-bucket cache-lifetime controls still gets the one-hour lifetime, rather than the flag silently having no effect.

4. Setup reports per client what it did with the flag. A client with no prompt-cache-lifetime control is named in the output along with the reason, and this is reported as information rather than as a failure. Naming such a client explicitly via --client behaves the same way and does not error.

5. The extended lifetime is off unless the flag is passed. Passing --extended-ttl-cache=false removes a previously applied extended lifetime from every client that had it, returning them to their default.

6. The setting is persisted, so a later plain thv llm setup re-applies it rather than silently dropping it. It can be set without running setup via thv llm config set, and its current state is visible in thv llm config show.

7. thv llm teardown removes the extended lifetime along with the rest of the client's gateway configuration.

8. When --extended-ttl-cache is combined with --bedrock-compat, setup applies the setting and warns that it may not take effect, because part of the one-hour request travels in a beta header that a Bedrock-backed gateway can reject or strip, and because one-hour support on Bedrock varies by model. The warning tells the user how to confirm whether it took effect.

9. When a client's existing configuration forces the five-minute lifetime and would override this request, setup warns that the extended lifetime cannot take effect and names the setting responsible, instead of writing configuration that provably does nothing.

Engineering discretion

• Printing the exact command a developer can run to confirm which cache lifetime their requests actually used, alongside the warnings in criteria 8 and 9.
• Expressing per-client support for a cache-lifetime control declaratively in the client registry, so a future client gains support by declaring it rather than by changing the reporting logic.

Out of scope

• Configuring the gateway itself to forward the beta header that carries the one-hour request. That is gateway-side configuration, not a ToolHive CLI concern.
• Cache-lifetime configuration for clients that expose no control (Claude Desktop, Codex, Gemini CLI, Cursor, VS Code, Xcode). They are reported as unsupported, not worked around.
• Any equivalent change in mecatl, which is tracked separately and already handled.
• Offering a choice of lifetime values. The flag is a boolean opt-in to the longer lifetime, not a --cache-ttl=5m|1h selector.
• Measuring, reporting, or attributing the realized cache savings.
• Repairing the Cursor and VS Code gateway configurations, which write keys those tools no longer read (Cursor LLM gateway config is a no-op: writes settings Cursor never reads #6293, VS Code LLM gateway config uses Copilot keys the extension no longer reads #6294). Those clients have no cache-lifetime control either way.

Additional context

• Origin: internal discussion of Anthropic cache reuse through the gateway measuring 27 to 53% of the rate seen on the one-hour default, with an estimated $5,033/month gross and $3,332/month net at full parity.
• Claude Code prompt caching reference, including the TTL precedence order and the two request buckets: https://code.claude.com/docs/en/prompt-caching
• Cache write pricing: 1.25x base input for five-minute, 2x for one-hour; reads are 0.1x on both.
• Related, not overlapping: Cursor LLM gateway config is a no-op: writes settings Cursor never reads #6293 and VS Code LLM gateway config uses Copilot keys the extension no longer reads #6294 (Cursor and VS Code gateway configs are no-ops); the same concern in mecatl is tracked in that repository.