Skip to content

Commit 12770e8

Browse files
fix: preserve cache_creation fields in responses API usage transform
The _transform_chat_completion_usage_to_responses_usage method drops Anthropic cache creation fields (cache_creation_input_tokens, cache_read_input_tokens, cache_creation.ephemeral_5m/1h_input_tokens) when converting chat-completion Usage to ResponseAPIUsage. This causes the inference proxy to see zero cache creation tokens, resulting in unbilled cache creation on /v1/responses. Fix: use setattr to add these fields as extras on ResponseAPIUsage (which supports extra fields via BaseLiteLLMOpenAIResponseObject's extra="allow" config), so they survive serialization and can be extracted downstream. Co-authored-by: Cursor <cursoragent@cursor.com>
1 parent f5fe620 commit 12770e8

1 file changed

Lines changed: 20 additions & 0 deletions

File tree

litellm/responses/litellm_completion_transformation/transformation.py

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2147,6 +2147,26 @@ def _transform_chat_completion_usage_to_responses_usage(
21472147
**output_details_dict
21482148
)
21492149

2150+
# Preserve Anthropic cache creation fields for downstream billing extraction.
2151+
# These are not part of the OpenAI ResponseAPIUsage schema but
2152+
# BaseLiteLLMOpenAIResponseObject has extra="allow", so setattr ensures
2153+
# they survive serialization and can be extracted by the inference proxy
2154+
# before the lossy api.Usage unmarshal.
2155+
if hasattr(usage, "cache_creation_input_tokens") and usage.cache_creation_input_tokens:
2156+
setattr(response_usage, "cache_creation_input_tokens", usage.cache_creation_input_tokens)
2157+
if hasattr(usage, "cache_read_input_tokens") and usage.cache_read_input_tokens:
2158+
setattr(response_usage, "cache_read_input_tokens", usage.cache_read_input_tokens)
2159+
if hasattr(usage, "prompt_tokens_details") and usage.prompt_tokens_details is not None:
2160+
ptd = usage.prompt_tokens_details
2161+
if hasattr(ptd, "cache_creation_token_details") and ptd.cache_creation_token_details is not None:
2162+
cache_creation_dict: Dict[str, int] = {}
2163+
if ptd.cache_creation_token_details.ephemeral_5m_input_tokens is not None:
2164+
cache_creation_dict["ephemeral_5m_input_tokens"] = ptd.cache_creation_token_details.ephemeral_5m_input_tokens
2165+
if ptd.cache_creation_token_details.ephemeral_1h_input_tokens is not None:
2166+
cache_creation_dict["ephemeral_1h_input_tokens"] = ptd.cache_creation_token_details.ephemeral_1h_input_tokens
2167+
if cache_creation_dict:
2168+
setattr(response_usage, "cache_creation", cache_creation_dict)
2169+
21502170
return response_usage
21512171

21522172
@staticmethod

0 commit comments

Comments
 (0)