Release 3.4.0 (#92)

Release 3.4.0: thinkwatch-core v0.69.0 (reasoning kept across formats, Gemini call signatures, tool calls written into text) and the forwarded-session fix.

Author: fylorn

CHANGELOG.md

@@ -11,6 +11,78 @@ target.
 
 ## [Unreleased]
 
+## [3.4.0] — 2026-10-11
+
+Conversations with reasoning models behind Chat-format upstreams keep their
+reasoning from turn to turn, Gemini's tool loops keep their thought
+signatures through a conversion, and a tool call that a Chat-format model
+writes into its answer text reaches the caller as a tool call. The
+thinkwatch-core crates move from v0.68.0 to v0.69.0.
+
+### Read before upgrading
+
+- **Converted answers from Chat-format upstreams can carry tool calls that
+  arrived as text before.** Self-hosted and relayed Chat-format models
+  (DeepSeek, GLM, Qwen, Llama and others) sometimes write a tool call into
+  `content` in the template they were trained on instead of answering with
+  `tool_calls`. On a route that converts the request to the Chat format,
+  such a call to a tool the request defined now reaches the caller as a
+  structured tool call, which the caller executes, where it used to be a
+  tagged piece of prose the caller showed and did nothing with. Tool-call
+  inspection sees these calls like any other and refuses or cuts them under
+  the same rules; the request log's response body records them as tool
+  calls, and a recovered call ends the answer with the tool-use stop reason
+  instead of `stop`. A request that defined no tools, a name the request
+  did not define, and a bare JSON object in prose stay text. A Chat caller
+  on a Chat-format route is forwarded as sent and unchanged. No setting,
+  schema, API route, Helm value or Redis key changes.
+
+### Fixed
+
+- **Gemini thought signatures are not counted as answer text.** When a
+  request's usage has to be estimated (`usage_estimated`), the
+  `thoughtSignature` on a Gemini thought or function call — an opaque blob
+  of up to several kilobytes — was counted as output text; it is left out
+  now, as the other formats' reasoning signatures were.
+
+### Changed
+
+- thinkwatch-core crates (tw-bedrock, tw-breaker, tw-dialect, tw-guard)
+  v0.68.0 → v0.69.0. Only tw-dialect changes:
+  - **Reasoning from a Chat-format upstream round-trips** (tw-dialect).
+    Reasoning a Chat-format upstream returned (`reasoning_content`, or
+    `reasoning` as OpenRouter and vLLM call it) was dropped from the next
+    turn's request on every route, so a Claude Code or Codex conversation
+    with tools against DeepSeek, which requires the reasoning of a
+    tool-calling turn on the following requests of that turn, lost it every
+    turn. Such reasoning is now carried in the caller's own format with a
+    `tw1.ch.` signature, read back on the next turn, and written as
+    `reasoning_content` only to a Chat-format upstream; to an upstream of
+    another format it is left out as before, and reasoning another vendor
+    signed is still not sent to a Chat-format upstream. Consecutive
+    assistant messages are merged before a request is written in the Chat
+    format, so a Responses caller's reasoning, text and tool call of one
+    turn arrive as one message. The reasoning history a request carries
+    back counts as its input, so the input tokens of those turns grow by
+    it, as the upstream charges them.
+  - **Gemini thought signatures on function calls are kept** (tw-dialect).
+    Gemini with thinking puts a turn's thought signature on the first
+    `functionCall` part, and the conversion only read signatures from
+    thought parts, so a converted Gemini tool loop lost the signature and
+    with it the model's reasoning continuity. The signature on a function
+    call is now carried through a conversion and written back on the
+    `functionCall` part; a call with no real signature gets the
+    `skip_thought_signature_validator` placeholder as before. A Gemini
+    request forwarded as sent that carries a signature from another route
+    on a `functionCall` part keeps the part with the placeholder, where the
+    part was removed and the tool loop broke.
+  - **Tool calls written into a Chat-format upstream's answer text are
+    recognised** (tw-dialect): see Read before upgrading. In this edition
+    the answer assembled for the request log and the cache of a Chat
+    caller's stream forwarded as sent to a Chat-format upstream keeps such
+    a call as the text the caller received, rather than the structured call
+    the conversion would have made of it.
+
 ## [3.3.1] — 2026-10-10
 
 Streams are logged as they ended. A caller that hangs up once it has read
@@ -1696,7 +1768,8 @@ unreleased builds should: stop the gateway, run `db/schema.sql`
 against PostgreSQL, restart against this tag. The schema is
 idempotent end-to-end, so the apply is safe to repeat.
 
-[Unreleased]: https://github.com/ThinkWatchProject/ThinkWatch/compare/v3.3.1...HEAD
+[Unreleased]: https://github.com/ThinkWatchProject/ThinkWatch/compare/v3.4.0...HEAD
+[3.4.0]: https://github.com/ThinkWatchProject/ThinkWatch/releases/tag/v3.4.0
 [3.3.1]: https://github.com/ThinkWatchProject/ThinkWatch/releases/tag/v3.3.1
 [3.3.0]: https://github.com/ThinkWatchProject/ThinkWatch/releases/tag/v3.3.0
 [3.2.1]: https://github.com/ThinkWatchProject/ThinkWatch/releases/tag/v3.2.1

Cargo.lock

@@ -10,7 +10,7 @@ members = [
 ]
 
 [workspace.package]
-version = "3.3.1"
+version = "3.4.0"
 edition = "2024"
 # The MSRV: the newest `rust-version` among the locked dependencies
 # (sqlx 0.9 and the aws-smithy crates under aws-sigv4). Without it,
@@ -70,10 +70,10 @@ opt-level = 3
 # never re-exported through a local shim. And the reverse: something only
 # this side uses (the at-rest crypto, IMDSv2 credentials, the gateway error)
 # lives here, not in core.
-tw-bedrock = { git = "https://github.com/ThinkWatchProject/ThinkWatch-Core.git", tag = "v0.68.0" }
-tw-breaker = { git = "https://github.com/ThinkWatchProject/ThinkWatch-Core.git", tag = "v0.68.0" }
-tw-dialect = { git = "https://github.com/ThinkWatchProject/ThinkWatch-Core.git", tag = "v0.68.0" }
-tw-guard = { git = "https://github.com/ThinkWatchProject/ThinkWatch-Core.git", tag = "v0.68.0" }
+tw-bedrock = { git = "https://github.com/ThinkWatchProject/ThinkWatch-Core.git", tag = "v0.69.0" }
+tw-breaker = { git = "https://github.com/ThinkWatchProject/ThinkWatch-Core.git", tag = "v0.69.0" }
+tw-dialect = { git = "https://github.com/ThinkWatchProject/ThinkWatch-Core.git", tag = "v0.69.0" }
+tw-guard = { git = "https://github.com/ThinkWatchProject/ThinkWatch-Core.git", tag = "v0.69.0" }
 
 # Web framework
 axum = { version = "0.8", features = ["macros", "ws"] }

Cargo.toml

@@ -357,6 +357,13 @@ impl Outbound {
     /// compaction would be recorded as a failed one. A request the
     /// conversion layer cannot read at all can still be forwarded, and its
     /// answer is assembled knowing only the model and whether it streams.
+    ///
+    /// A Chat upstream can write a tool call into its answer text instead
+    /// of `tool_calls`. A converted answer turns such a call, when the
+    /// request defined the tool, into a structured one (see
+    /// `tw_dialect::chat::text_calls`); a forwarded answer reaches the
+    /// caller as the upstream wrote it, text included, and is assembled as
+    /// that (`Session::keep_text_calls`).
     fn forwarded_session(&self, body: &Value, model: &str, target: &Target) -> Session {
         let client = self.surface.dialect;
         let request =
@@ -368,7 +375,9 @@ impl Outbound {
                     ..Default::default()
                 },
             };
-        tw_dialect::convert::encode(&request, target).session
+        tw_dialect::convert::encode(&request, target)
+            .session
+            .keep_text_calls()
     }
 
     /// Address the request to `protocol`, naming `model` upstream.
@@ -1273,6 +1282,63 @@ mod tests {
         assert!(converted.convert.as_ref().unwrap().is_compaction());
     }
 
+    /// A Chat upstream that writes a tool call into its text: a converted
+    /// answer carries it as a structured call, a forwarded one is assembled
+    /// as the caller received it — text.
+    #[test]
+    fn a_tool_call_written_into_text_is_assembled_as_the_caller_received_it() {
+        const STREAM: &str = concat!(
+            "data: {\"id\":\"c\",\"object\":\"chat.completion.chunk\",\"choices\":[{\"index\":0,",
+            "\"delta\":{\"role\":\"assistant\",\"content\":\"<tool_call>{\\\"name\\\": \\\"ls\\\", ",
+            "\\\"arguments\\\": {\\\"path\\\": \\\"/\\\"}}</tool_call>\"}}]}\n\n",
+            "data: {\"id\":\"c\",\"object\":\"chat.completion.chunk\",\"choices\":[{\"index\":0,",
+            "\"delta\":{},\"finish_reason\":\"stop\"}]}\n\n",
+            "data: [DONE]\n\n"
+        );
+        let tools = serde_json::json!([{
+            "type": "function",
+            "function": {"name": "ls", "parameters": {"type": "object"}}
+        }]);
+        let assemble = |session: &Session| -> Value {
+            let mut c = session.collector();
+            c.process(STREAM.as_bytes());
+            serde_json::from_slice(&c.finish().unwrap()).unwrap()
+        };
+
+        let ask = serde_json::json!({
+            "model": "m", "stream": true, "tools": tools,
+            "messages": [{"role": "user", "content": "list /"}]
+        });
+        let forwarded = outbound(CHAT, "/v1/chat/completions", ask, None)
+            .address(UpstreamProtocol::OpenAiChat, "m", false)
+            .unwrap_or_else(|e| panic!("{e:?}"));
+        assert!(forwarded.convert.is_none());
+        let message = &assemble(&forwarded.collect)["choices"][0]["message"];
+        assert!(
+            message["content"].as_str().unwrap().contains("<tool_call>"),
+            "{message}"
+        );
+        assert!(message.get("tool_calls").is_none(), "{message}");
+
+        let ask = serde_json::json!({
+            "model": "m", "stream": true, "max_tokens": 100,
+            "tools": [{"name": "ls", "input_schema": {"type": "object"}}],
+            "messages": [{"role": "user", "content": "list /"}]
+        });
+        let converted = outbound(MESSAGES, "/v1/messages", ask, None)
+            .address(UpstreamProtocol::OpenAiChat, "m", false)
+            .unwrap_or_else(|e| panic!("{e:?}"));
+        let content = &assemble(converted.convert.as_ref().unwrap())["content"];
+        assert!(
+            content
+                .as_array()
+                .unwrap()
+                .iter()
+                .any(|b| b["type"] == "tool_use" && b["name"] == "ls"),
+            "{content}"
+        );
+    }
+
     /// Only breakpoints the conversion added may be taken out when an
     /// upstream refuses them; the caller's own are its decision.
     #[test]

crates/gateway/src/proxy/generate.rs

@@ -91,6 +91,8 @@ fn text_len(v: &Value, skip: &[&str]) -> usize {
         "call_id",
         "tool_call_id",
         "signature",
+        "thoughtSignature",
+        "thought_signature",
         "encrypted_content",
     ];
     match v {
@@ -175,6 +177,15 @@ mod tests {
             ],
         });
         assert_eq!(answer_tokens(body.to_string().as_bytes()), 2);
+        // Gemini's, on a thought and on the function call it led to.
+        let body = json!({
+            "candidates": [{"content": {"parts": [
+                {"text": "abcd", "thought": true, "thoughtSignature": "x".repeat(4000)},
+                {"functionCall": {"name": "ls", "args": {}}, "thoughtSignature": "x".repeat(4000)},
+                {"functionCall": {"name": "ls", "args": {}}, "thought_signature": "x".repeat(4000)}
+            ]}}],
+        });
+        assert_eq!(answer_tokens(body.to_string().as_bytes()), 2);
     }
 
     #[test]

crates/gateway/src/usage_estimate.rs

@@ -2,8 +2,8 @@ apiVersion: v2
 name: think-watch
 description: Enterprise AI API Gateway & MCP Management Platform
 type: application
-version: 3.3.1
-appVersion: "3.3.1"
+version: 3.4.0
+appVersion: "3.4.0"
 keywords:
   - ai
   - gateway

deploy/helm/think-watch/Chart.yaml

@@ -1,7 +1,7 @@
 {
   "name": "web",
   "private": true,
-  "version": "3.3.1",
+  "version": "3.4.0",
   "type": "module",
   "packageManager": "pnpm@12.9.1",
   "scripts": {