Skip to content

Use prompt caching in the Anthropic machinery #21310

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jeenadeepak wants to merge 3 commits into
base: main
Choose a base branch
from
+15 −1
Open

Use prompt caching in the Anthropic machinery #21310

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f085007
Use prompt caching in the Anthropic machinery
jeenadeepak Aug 25, 2026
48fe795
chore(pre-commit): apply code formatting
pre-commit-ci-lite[bot] Aug 25, 2026
fbde9d7
Merge branch 'main' into anthropic-prompt-caching
jeenadeepak Aug 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/changes.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ Weblate 2026.9

* :ref:`addon-weblate.gettext.xgettext` now accepts multiple custom keywords (newline-separated) passed to xgettext via ``--keyword``, enabling extraction from different function names.
* Screenshot images are now cached in browsers to reduce repeated downloads.
* Anthropic machinery now uses prompt caching to reduce the API costs of automatic translation.
* Administrators can now find removed accounts by their former e-mail address in the audit log until :setting:`AUDITLOG_EXPIRY`.
* Deployment checks and the performance report now detect slow filesystem metadata access in data and cache directories.
* Clarified the instance-wide impact of roles containing site-wide permissions, including :ref:`site-wide user management <site-wide-user-management>`.
Expand Down
8 changes: 7 additions & 1 deletion weblate/machinery/anthropic.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,13 @@ def get_chat_payload(
return {
"model": model,
"max_tokens": self.settings.get("max_tokens", 4096),
"system": prompt,
"system": [
{
"type": "text",
"text": prompt,
"cache_control": {"type": "ephemeral"},

@chatgpt-codex-connector chatgpt-codex-connector Bot Aug 25, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Gate caching for one-off Anthropic requests

When the system prefix meets Anthropic's minimum cacheable length but no identical request follows within the five-minute TTL, this unconditional breakpoint creates a cache write billed at 1.25× the normal input rate without any discounted cache read. Because this payload is used for ordinary editor suggestions and settings validation as well as sequential automatic-translation batches, installations making sporadic requests—or jobs completed in one batch—will pay more despite the changelog's cost-reduction claim; restrict caching to repeated/bulk traffic or make it configurable.

Useful? React with 👍 / 👎.

}
],
"messages": [
{"role": "user", "content": previous_content},
{"role": "assistant", "content": previous_response},
Expand Down
12 changes: 12 additions & 0 deletions weblate/machinery/tests.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2945,8 +2945,8 @@
with self.subTest(test_case["name"]):
machine = self.MACHINE_CLS(
{
"key": test_case["key"],

Check failure on line 2948 in weblate/machinery/tests.py

View workflow job for this annotation

GitHub Actions / mypy 

Incompatible types (expression has type "object", TypedDict item "key" has type "str")
"url": test_case["url"],

Check failure on line 2949 in weblate/machinery/tests.py

View workflow job for this annotation

GitHub Actions / mypy 

Incompatible types (expression has type "object", TypedDict item "url" has type "str")
}
)
self.assertEqual(machine.api_base_url, test_case["base_url"])
Expand Down Expand Up @@ -3615,7 +3615,7 @@
)

def test_language_instructions_empty_initial_renders_blank(self) -> None:
for initial in (

Check failure on line 3618 in weblate/machinery/tests.py

View workflow job for this annotation

GitHub Actions / mypy 

Need type annotation for "initial"
{},
{"language_instructions": None},
{"language_instructions": {}},
Expand Down Expand Up @@ -6239,15 +6239,15 @@
try:
highlights = highlight_string(unit.source, unit)
except Exception as error: # pragma: no cover - diagnostic path
highlights = f"{error.__class__.__name__}: {error}"

Check failure on line 6242 in weblate/machinery/tests.py

View workflow job for this annotation

GitHub Actions / mypy 

Incompatible types in assignment (expression has type "str", variable has type "list[Highlight]")
try:
machine_highlights = list(machine.get_highlights(unit.source, unit))
except Exception as error: # pragma: no cover - diagnostic path
machine_highlights = f"{error.__class__.__name__}: {error}"

Check failure on line 6246 in weblate/machinery/tests.py

View workflow job for this annotation

GitHub Actions / mypy 

Incompatible types in assignment (expression has type "str", variable has type "list[Any]")
try:
placeholder_values = unit.all_flags.get_value_raw("placeholders")
except Exception as error: # pragma: no cover - diagnostic path
placeholder_values = f"{error.__class__.__name__}: {error}"

Check failure on line 6250 in weblate/machinery/tests.py

View workflow job for this annotation

GitHub Actions / mypy 

Incompatible types in assignment (expression has type "str", variable has type "tuple[str | Pattern[str], ...]")

return json.dumps(
{
Expand Down Expand Up @@ -7780,6 +7780,18 @@
)


@http_mock.activate
def test_system_prompt_cache_control(self) -> None:
self.mock_response()
self.assert_translate(
self.SUPPORTED, self.SOURCE_TRANSLATED, self.EXPECTED_LEN
)
payload = json.loads(http_mock.calls[0].request.content)
self.assertEqual(
payload["system"][0]["cache_control"], {"type": "ephemeral"}
)


class AnthropicCustomModelTranslationTest(AnthropicTranslationTest):
CONFIGURATION: ClassVar[SettingsDict] = {
"key": "test-api-key",
Expand Down Expand Up @@ -8610,7 +8622,7 @@

for service in third_party_services:
self.assertTrue(service.sends_data_to_third_party, service.name)
for service in local_services:

Check failure on line 8625 in weblate/machinery/tests.py

View workflow job for this annotation

GitHub Actions / mypy 

Incompatible types in assignment (expression has type "type[CyrTranslitTranslation] | type[DummyTranslation] | type[WeblateMemory] | type[WeblateTranslation]", variable has type "type[AlibabaTranslation] | type[AnthropicTranslation] | type[ApertiumAPYTranslation] | type[AWSTranslation] | type[BaiduTranslation] | <19 more items>")
self.assertFalse(service.sends_data_to_third_party, service.name)

@override_settings(OFFER_HOSTING=False)
Expand Down
Loading