Skip to content

feat(agenticclaude): expose cache_creation_input_tokens in message.Extra - #978

Open
sin-bufan wants to merge 3 commits into
cloudwego:mainfrom
sin-bufan:feat/agenticclaude-cache-and-vertex-creds
Open

feat(agenticclaude): expose cache_creation_input_tokens in message.Extra#978
sin-bufan wants to merge 3 commits into
cloudwego:mainfrom
sin-bufan:feat/agenticclaude-cache-and-vertex-creds

Conversation

@sin-bufan

@sin-bufan sin-bufan commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Summary

Port cache_creation_input_tokens from the legacy claude adapter to agenticclaude, following the pattern established in #929.

Anthropic bills prompt cache reads (~10% of base rate) and writes (~125% of base rate) at very different prices. The read side is already exposed through the standard PromptTokenDetails.CachedTokens path. The creation side (cache_creation_input_tokens) has no standard schema equivalent, so it is surfaced in message.Extra:

  • Key: _eino_claude_cache_creation_input_tokens (same key as the legacy claude adapter)
  • Set in: toAgenticMessage (Generate) and toMessageStreamingChunk (message_delta)
  • Read via: agenticclaude.GetCacheCreationInputTokens(msg *schema.AgenticMessage) (int, bool)
  • Streaming: Anthropic reports once per request on message_start or message_delta. int has built-in useLast concat, so ConcatAgenticMessages correctly preserves the value without accumulation.
  • Zero activity: When CacheCreationInputTokens == 0, no Extra entry is written.

Design decisions

  • Same Extra key as legacy adapter — Downstream consumers (e.g. Langfuse cost tracking) do not need to distinguish between legacy and agentic adapters.
  • Only cache_creation in Extracache_read is NOT duplicated because it's already in the standard PromptTokenDetails.CachedTokens path.
  • int type, not a struct — Simpler than a typed struct; no need to register concat func since int has built-in support.

Changes

File Change
consts.go Add exported KeyOfCacheCreationInputTokens constant
convertor.go Set cache_creation in toAgenticMessage (Generate path)
event_convertor.go Set cache_creation in toMessageStreamingChunk (message_delta)
message_extra.go New: GetCacheCreationInputTokens() getter
message_extra_test.go New: tests for getter, Generate path, streaming delta path

Test plan

  • TestGetCacheCreationInputTokens — nil message, nil Extra, key missing, key present
  • TestCacheCreationInputTokens_SetInConvOutputMessage — zero and nonzero paths
  • TestCacheCreationInputTokens_SetInStreamDelta — zero and nonzero paths
  • All existing tests pass (go test ./...)

Related

Port cache_creation_input_tokens from the legacy claude adapter to
agenticclaude, following the pattern established in cloudwego#929:

- Add KeyOfCacheCreationInputTokens constant (same key as legacy adapter)
- Set in toAgenticMessage (Generate) and streamConverter (message_delta)
- Add GetCacheCreationInputTokens() getter for downstream consumers
- Zero value: no Extra entry is written when CacheCreationInputTokens == 0

Co-authored-by: Cursor <cursoragent@cursor.com>
@sin-bufan
sin-bufan force-pushed the feat/agenticclaude-cache-and-vertex-creds branch from 382f254 to b4c6212 Compare August 28, 2026 08:44
@sin-bufan sin-bufan changed the title feat(agenticclaude): expose cache_creation_input_tokens in message.Extra and add Vertex ServiceAccountJSON feat(agenticclaude): expose cache_creation_input_tokens in message.Extra Aug 28, 2026
Comment thread components/model/agenticclaude/consts.go Outdated
Comment thread components/model/agenticclaude/message_extra.go
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants