[Bug]: Agent conversations under high context pressure permanently break Anthropic prompt caching (tool-result re-truncation shifts the cache prefix on every turn)

[sisve]

What happened?

A long-running agent conversation that accumulates large tool outputs (MCP tool calls returning tens-to-hundreds of KB of JSON) permanently loses Anthropic prompt-cache reuse once the conversation's context pressure crosses the SDK's masking/truncation threshold (~80%). This is not a one-time cache miss — it recurs on every subsequent turn for the rest of the conversation's life, because the SDK recomputes, from the conversation's current total size, how much of an already-completed, historical tool result to keep. As a result, the same historical tool result gets a different truncated length on nearly every call. Since Anthropic's prompt cache is prefix-based, any byte change to an early message invalidates the cached prefix for everything after it, forcing a full, expensive cache rewrite on every affected turn.

Impact observed on one real conversation over one day: 144 full cache-write events; ~18.7M tokens written to the 1-hour cache tier (2x base input price) versus ~35.8M tokens read from cache. On Claude Opus 4.6 pricing that's roughly $187 of a single day's ~$225 total spend, attributable to this one mechanism, in one user's one conversation.

Root cause:

1. Diffed two Anthropic API request payloads sent 11 seconds apart within the same conversation. Confirmed via LibreChat's own message tree in MongoDB that the conversation has zero forks/regenerations (fully linear parentMessageId chain) — this is not a user editing or regenerating a message.
2. Every message in the payload was byte-identical between the two calls except one: an early tool_result message (the output of an MCP tool call from much earlier in the conversation) shrank from 312,301 to 305,977 bytes — specifically, one content block inside it shrank from 280,055 to 273,731 bytes.
3. Confirmed via LibreChat's own MongoDB messages collection that the underlying tool output is durably stored in full (266,534 chars in one checked instance) — so the tool is not being re-executed; the same stored output is simply being truncated to a different length on each payload build.
4. Traced this to @librechat/agents' messages/prune.cjs:
   • contextPressure = calibratedTotalTokens / pruningBudget is recomputed on every LLM call.
   • Once contextPressure >= PRESSURE_THRESHOLD_MASKING (0.8), maskConsumedToolResults and/or preFlightTruncateToolResults (using PRESSURE_BANDS when summarization is disabled, or a rawSpaceEffectiveMax-derived budget when it's enabled) re-truncates old tool outputs to fit the current remaining budget.
   • Because calibratedTotalTokens shifts every turn as the conversation grows, the truncation applied to a fixed, already-completed historical tool result differs nearly every call.
5. Because the affected message sits early in a long conversation, and Anthropic's cache is prefix-based (a hit requires exact-byte match of everything up to the marked breakpoint), this re-truncation invalidates the entire downstream cached prefix on every affected turn — forcing a full-conversation cache rewrite repeatedly, for the rest of that conversation's life once it crosses the pressure threshold (which it never un-crosses, since conversations only grow).

Relationship to existing reports: same symptom as #11615 and #12209 (huge cache-write token counts, near-zero cache hits on agent conversations), but neither thread's working theories (cache-breakpoint placement, memory-injection ordering) match this specific mechanism.

Note on a related fix already shipped: v3.7.5 (danny-avila/agents#473, "align compaction with provider cache prefixes") changed exactly this area of the codebase — it aligns the summarization/"compaction" self-summarize request path with provider cache prefixes, and its own before/after numbers show a compaction request going from 0 cached tokens to a full cache hit. That's the right direction, but it's unclear from the PR description whether the fix also covers the plain pre-flight truncation path (preFlightTruncateToolResults / maskConsumedToolResults in prune.cjs) exercised here — which any conversation without an explicit summarization.enabled: false will still hit, since enabled defaults to true.

What I expected to happen: once a tool result has been included in a cache write, its serialized bytes should stay stable across subsequent turns (or compaction should replace it with a new message rather than mutate it in place), so the existing cache prefix stays valid instead of being invalidated on every turn.

Version Information

I am using a forked v0.8.8-rc1 build, so the built in dialog is not populated.

• LibreChat: v0.8.8-rc1 (self-hosted)
• @librechat/agents: 3.4.7
• Endpoint: agents, provider Anthropic (Claude Opus 4.6), MCP tools attached
• Summarization: not explicitly configured (no summarization: block at agent or app level) — enabled resolves to the SDK's default of true

Steps to Reproduce

1. Start a LibreChat agent conversation using an MCP tool that returns outputs in the 100–300 KB range.
2. Continue the conversation across enough turns/tool calls that total context usage crosses roughly 80% of the model's context window.
3. Capture two Anthropic API request payloads from consecutive turns (or even sub-turns within a single agent tool-use loop) and diff them message-by-message.
4. Observe: an early tool_result message (well before the active turn) differs in byte length between the two payloads despite no edit/regeneration having occurred, and the affected request's cache_creation_input_tokens shows a full-size rewrite rather than a cache hit for that portion of the conversation.

What browsers are you seeing the problem on?

No response

Relevant log output

(Left empty on purpose.)

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct