Release 3.3.0 (#89)

See CHANGELOG.md [3.3.0].

Author: fylorn

CHANGELOG.md

@@ -11,6 +11,106 @@ target.
 
 ## [Unreleased]
 
+## [3.3.0] — 2026-10-09
+
+Requests converted for Claude now use prompt caching, and Codex works
+through routes that convert its requests to another format: its tools,
+its history and its compactions arrive. A Responses request that only
+OpenAI can read — one that continues a conversation kept on OpenAI's
+servers, or carries a compaction OpenAI wrote — now goes to a route that
+speaks Responses instead of being refused. The thinkwatch-core crates move
+from v0.65.0 to v0.67.1.
+
+### Read before upgrading
+
+- **Converted requests to Claude are cached, and billed as cache traffic.**
+  A request converted for an Anthropic-format upstream, or for a Claude
+  model on Bedrock that AWS lists for prompt caching (Claude 3.5 Sonnet v2,
+  Claude 3.7 Sonnet, and Claude 4.5 and later), from a client that marks no
+  cache breakpoints — Codex, Chat and Gemini clients, and an Anthropic
+  client whose request goes to Bedrock without `cache_control` — now gets
+  them at the end of the tools, the system prompt and the last two user
+  turns, with the 5-minute lifetime.
+  - The upstream bills the prefix a request writes at its cache-write rate
+    (1.25× input at Anthropic) and what the next turns read back at its
+    cache-read rate (0.1×). ThinkWatch prices and counts them the same
+    way: cost, budgets and `tokens` rate limits weigh a cache write by the
+    model's `cache_write_weight` and a read by its `cache_read_weight`,
+    1.25× and 0.1× its input weight when unset. Set them on the model to
+    match what your upstream charges.
+  - A conversation of several turns costs less. A one-off request with a
+    long prompt costs up to a quarter more on its input.
+  - Prompt token counts (`input_tokens` on the log rows, quotas) include
+    cached input in full, as before.
+  - Requests that mark their own breakpoints, such as Claude Code's, and
+    requests forwarded in their own format are unchanged.
+  - An upstream that refuses the added breakpoints — a `400` that names
+    `cache_control`, `cachePoint` or prompt caching, as an
+    Anthropic-compatible endpoint that does not know them answers — gets
+    the request once more without them, and later requests to it for that
+    model leave them out from the start. Each instance remembers this
+    until a provider or model is changed or it restarts. Breakpoints a
+    client marked itself are never taken out, and the probes that check a
+    route's API format go without breakpoints.
+- **Rolling back with Codex sessions.** A Codex session that compacted
+  through a converted route on 3.3.0 carries a compaction that 3.2.1
+  refuses, so after a rollback, or on a 3.2.1 instance during the rollout,
+  it cannot continue and needs a new session.
+
+Nothing else needs action: no setting, schema, API route or Redis key
+changes.
+
+### Fixed
+
+- **Responses requests that only OpenAI can read.** A Responses request
+  that continues a conversation kept on OpenAI's servers
+  (`previous_response_id`, `conversation`, `prompt`, `background`), points
+  at a stored item (`item_reference`) or carries a compaction OpenAI wrote
+  was refused with `400`, even on a route that speaks Responses and would
+  have forwarded it as sent: every request was decoded for its usage
+  estimate, and one that could not be decoded was refused. Codex sends such
+  a compaction in every request after compacting a session through an
+  OpenAI upstream, so the session could not go on; and a WebSocket turn
+  naming a response other than the connection's last one, meant to go
+  upstream as sent, was refused too. These requests now go to the model's
+  routes that speak Responses, and only a model with none refuses them,
+  with the reason. Their input estimate counts the request's text.
+
+### Changed
+
+- thinkwatch-core crates (tw-bedrock, tw-breaker, tw-dialect, tw-guard)
+  v0.65.0 → v0.67.1. Only tw-dialect, the format conversion, changes:
+  - **Prompt caching for Claude** on converted requests (see Read before
+    upgrading).
+  - **Codex tools.** For some models Codex declares every tool in an
+    `additional_tools` input item and sends no top-level `tools`. Those
+    items were dropped, so a route converting to Anthropic, Chat, Gemini
+    or Bedrock sent no tools and Codex could not read files or run
+    commands. Every tool now arrives, and tool calls come back under the
+    names and kinds Codex declared. Local shell calls, tool search,
+    reasoning-effort changes and agent messages in the history are
+    converted instead of dropped.
+  - **Codex compaction on converted routes.** Codex compacts a long
+    session through a provider named `OpenAI` by asking for exactly one
+    compaction item, which a converted route could not give, so compacting
+    failed. The upstream now writes a handoff summary, Codex receives it as
+    its compaction, and later requests carry it back. The summary request
+    is billed like any other. The summary travels in the item's
+    `encrypted_content` (`tw1.c.…`) base64-encoded, not encrypted; with
+    `audit.body_redact_pii` on, captured bodies have it redacted like the
+    rest of the body. OpenAI's own compactions still cannot be read by an
+    upstream of another format, and are refused saying so.
+  - **System messages in mid-conversation stay in place.** A `system` or
+    `developer` message after the conversation has started was added to
+    the system prompt on a converted route. It now stays where it was
+    given: a developer message for a Responses upstream, and a user turn
+    wrapped in `<system-reminder>` for Anthropic, Chat, Gemini and Bedrock
+    upstreams. The system prompt stays the same from one turn to the next,
+    which is what keeps the prompt cache working.
+  - **`verbosity`** (Chat) and **`text.verbosity`** (Responses) carry
+    across the conversion to GPT-5 and later models. Other models still
+    leave it out.
+
 ## [3.2.1] — 2026-10-09
 
 The Helm chart and the Compose file now pull the published images: they
@@ -1469,7 +1569,8 @@ unreleased builds should: stop the gateway, run `db/schema.sql`
 against PostgreSQL, restart against this tag. The schema is
 idempotent end-to-end, so the apply is safe to repeat.
 
-[Unreleased]: https://github.com/ThinkWatchProject/ThinkWatch/compare/v3.2.1...HEAD
+[Unreleased]: https://github.com/ThinkWatchProject/ThinkWatch/compare/v3.3.0...HEAD
+[3.3.0]: https://github.com/ThinkWatchProject/ThinkWatch/releases/tag/v3.3.0
 [3.2.1]: https://github.com/ThinkWatchProject/ThinkWatch/releases/tag/v3.2.1
 [3.2.0]: https://github.com/ThinkWatchProject/ThinkWatch/releases/tag/v3.2.0
 [3.1.0]: https://github.com/ThinkWatchProject/ThinkWatch/releases/tag/v3.1.0

Cargo.lock

@@ -10,7 +10,7 @@ members = [
 ]
 
 [workspace.package]
-version = "3.2.1"
+version = "3.3.0"
 edition = "2024"
 # The MSRV: the newest `rust-version` among the locked dependencies
 # (sqlx 0.9 and the aws-smithy crates under aws-sigv4). Without it,
@@ -70,10 +70,10 @@ opt-level = 3
 # never re-exported through a local shim. And the reverse: something only
 # this side uses (the at-rest crypto, IMDSv2 credentials, the gateway error)
 # lives here, not in core.
-tw-bedrock = { git = "https://github.com/ThinkWatchProject/ThinkWatch-Core.git", tag = "v0.65.0" }
-tw-breaker = { git = "https://github.com/ThinkWatchProject/ThinkWatch-Core.git", tag = "v0.65.0" }
-tw-dialect = { git = "https://github.com/ThinkWatchProject/ThinkWatch-Core.git", tag = "v0.65.0" }
-tw-guard = { git = "https://github.com/ThinkWatchProject/ThinkWatch-Core.git", tag = "v0.65.0" }
+tw-bedrock = { git = "https://github.com/ThinkWatchProject/ThinkWatch-Core.git", tag = "v0.67.1" }
+tw-breaker = { git = "https://github.com/ThinkWatchProject/ThinkWatch-Core.git", tag = "v0.67.1" }
+tw-dialect = { git = "https://github.com/ThinkWatchProject/ThinkWatch-Core.git", tag = "v0.67.1" }
+tw-guard = { git = "https://github.com/ThinkWatchProject/ThinkWatch-Core.git", tag = "v0.67.1" }
 
 # Web framework
 axum = { version = "0.8", features = ["macros", "ws"] }

Cargo.toml

@@ -7,6 +7,8 @@ edition.workspace = true
 tw-breaker = { workspace = true }
 tw-guard = { workspace = true }
 tw-bedrock = { workspace = true }
+# Reads the compaction summaries a conversion carries (see `pii`).
+tw-dialect = { workspace = true }
 axum = { workspace = true }
 sqlx = { workspace = true }
 fred = { workspace = true }
@@ -46,6 +48,3 @@ aws-sigv4 = { workspace = true }
 aws-credential-types = { workspace = true }
 http_1x = { package = "http", version = "1" }
 async-trait = "0.1"
-
-[dev-dependencies]
-tw-dialect = { workspace = true }

crates/common/Cargo.toml

@@ -14,6 +14,7 @@
 //! thinkwatch-core's (`tw_guard::redact`), the engine the desktop gateway
 //! redacts with.
 
+use std::borrow::Cow;
 use std::sync::Arc;
 
 use tw_guard::policy::RedactPolicy;
@@ -27,22 +28,70 @@ use tw_guard::redact::rules::RuleSet;
 /// JSON escapes are read, a custom rule's match stays inside one JSON
 /// string so the result is still JSON, base64 payloads are left alone,
 /// and so are the gateway's own placeholders (a captured answer still
-/// carries them).
+/// carries them). The one base64 payload that is read is a compaction
+/// summary a conversion carries (see [`redact_carried_summaries`]).
 pub fn redact_blob(rules: &RuleSet, input: &str) -> String {
     if rules.is_empty() {
         return input.to_string();
     }
-    let hits = tw_guard::redact::flow::hits(input, rules);
-    let mut out = input.to_string();
+    let input = redact_carried_summaries(rules, input);
+    let hits = tw_guard::redact::flow::hits(&input, rules);
+    let mut out = input.into_owned();
     for h in hits.iter().rev() {
-        out.replace_range(
-            h.bytes.clone(),
-            &format!("{{{{REDACTED_{}}}}}", h.rule.id()),
-        );
+        out.replace_range(h.bytes.clone(), &redacted(&h.rule));
     }
     out
 }
 
+fn redacted(rule: &tw_guard::redact::rules::Rule) -> String {
+    format!("{{{{REDACTED_{}}}}}", rule.id())
+}
+
+/// The summaries in `input` that a conversion carries, redacted.
+///
+/// A Codex compaction sent to an upstream of another format comes back as
+/// the summary that upstream wrote — the conversation restated, with its
+/// paths, commands and values — base64-encoded in the item's
+/// `encrypted_content` (`tw1.c.…`, see `tw_dialect::compaction`), and
+/// later requests carry it back. Unlike OpenAI's own compactions it is not
+/// encrypted, but as base64 the search above passes over it. Each one is
+/// read, redacted as plain text and written back the same way, so the
+/// body keeps its shape.
+fn redact_carried_summaries<'a>(rules: &RuleSet, input: &'a str) -> Cow<'a, str> {
+    const PREFIX: &str = tw_dialect::compaction::CARRIED_PREFIX;
+    if !input.contains(PREFIX) {
+        return Cow::Borrowed(input);
+    }
+    let mut out = String::with_capacity(input.len());
+    let mut rest = input;
+    while let Some(at) = rest.find(PREFIX) {
+        let start = at + PREFIX.len();
+        let end = rest[start..]
+            .find(|c: char| !(c.is_ascii_alphanumeric() || c == '-' || c == '_'))
+            .map_or(rest.len(), |n| start + n);
+        let carried = &rest[at..end];
+        out.push_str(&rest[..at]);
+        match tw_dialect::compaction::read(carried) {
+            Some(summary) => {
+                let hits = tw_guard::redact::flow::hits_plain(&summary, rules);
+                if hits.is_empty() {
+                    out.push_str(carried);
+                } else {
+                    let mut text = summary;
+                    for h in hits.iter().rev() {
+                        text.replace_range(h.bytes.clone(), &redacted(&h.rule));
+                    }
+                    out.push_str(&tw_dialect::compaction::carry(&text));
+                }
+            }
+            None => out.push_str(carried),
+        }
+        rest = &rest[end..];
+    }
+    out.push_str(rest);
+    Cow::Owned(out)
+}
+
 /// The at-rest redactor, hot-swapped with the outbound redaction policy.
 #[derive(Clone)]
 pub struct BlobRedactor {
@@ -133,6 +182,31 @@ mod tests {
         assert_eq!(v["b"], "keep", "{out}");
     }
 
+    #[test]
+    fn a_carried_compaction_summary_is_redacted_where_it_is() {
+        use tw_dialect::compaction::{carry, read};
+        let rules = only(&[("EMAIL", r"[\w.]+@[\w.]+")]);
+        let carried = carry("Reply to alice@example.com about src/lib.rs.");
+        let body = format!(
+            r#"{{"input":[{{"type":"compaction","encrypted_content":"{carried}"}},{{"role":"user","content":"bob@example.com"}}]}}"#
+        );
+        let out = redact_blob(&rules, &body);
+        let v: serde_json::Value = serde_json::from_str(&out).expect("still JSON");
+        let summary = read(v["input"][0]["encrypted_content"].as_str().unwrap());
+        assert_eq!(
+            summary.as_deref(),
+            Some("Reply to {{REDACTED_EMAIL}} about src/lib.rs."),
+            "{out}"
+        );
+        assert_eq!(v["input"][1]["content"], "{{REDACTED_EMAIL}}", "{out}");
+
+        // Nothing found in it, or not one of ours: left as it was.
+        let clean = format!(r#"{{"encrypted_content":"{}"}}"#, carry("tests pass"));
+        assert_eq!(redact_blob(&rules, &clean), clean);
+        let theirs = r#"{"encrypted_content":"gAAAAABoQ2xpZW50"}"#;
+        assert_eq!(redact_blob(&rules, theirs), theirs);
+    }
+
     #[test]
     fn the_policys_rules_are_the_ones_used_built_in_and_custom() {
         let key = "sk-ant-api03-AAAAAAAAAAAAAAAAAAAAAAAAAAAA";