fix(glm): vendor a minja-compatible chat template — tool calling works, and the #1254 mechanism was wrong - #1255
Merged
Conversation
…lling works Closes the tool path on all 18 GLM composes. Reported by @paulp83 (club-3090#1250) and independently upstream (ggml-org/llama.cpp#27754, 2026-09-07, unanswered). ⛔⛔ THE LINE IN THE ERROR IS THE CRASH SITE, NOT THE CAUSE. Two earlier readings of this bug were wrong — including one I committed in #1254 — and both are recorded in the patch README so nobody re-derives them: ❌ "minja cannot evaluate .items()" — GLM-4.7-Flash.jinja carries the identical construct, is exercised by llama.cpp's own autoparser tests, and passes. ❌ "the caps probe under-detects supports_object_arguments because the template iterates instead of indexing" — adding a named access changes nothing; the probe never gets that far. ACTUAL CAUSE: minja does not implement NUMERIC DOTTED ATTRIBUTE ACCESS (`x.0`). Jinja2 defines `x.0` as `x[0]`; minja parses it as a non-computed member whose property is a number literal, and throws. The GGUF's embedded template uses it in 4 places, so the jinja caps probe throws, infers NOTHING, and leaves EVERY capability false — including supports_object_arguments, which gates the JSON-string -> object conversion of tool arguments in chat.cpp. Arguments stay a string, and the autoparser then dies on `.items()` at line 163. That also explains supports_string_content:false, which no tools-only account covers and which was the clue that the earlier readings were wrong. The override is the vendor template with `.0.` -> `[0].` in 4 places plus a null-tool guard on the tools loop (the caps probe passes tools=[null], which trips `'function' in tool`). 5 lines. WIRE FORMAT UNCHANGED — tool calls still render <tool_call>NAME<arg_key>K</arg_key><arg_value>V</arg_value>…, which matters because changing it would stop the generated parser matching what the model emits. Verified without loading 140 GiB, using llama.cpp's own tooling: llama-template-analysis --template-file <template> embedded : tools/tool_calls/parallel/string_content ALL FALSE override : ALL TRUE llama-debug-template-parser (the real autoparser path): override yields tool_mode TAG_WITH_TAGGED, per_call_start '<tool_call>', full PEG + GBNF with the <arg_key>/<arg_value> rules. Reference Jinja2 3.1.2: original ≡ override byte-identical across 8 cases (two-call sort path, reasoning_content, typed content, list-of-outputs tool content, <think> split).⚠️ NOT live-booted. Static analysis + the engine's own autoparser, not a 140 GiB end-to-end serve. The drift guard asserts positively (caps must read true) rather than by absence, because a re-vendored template without the fix fails SILENTLY: boot green, /health green, tool calls 400.⚠️ Still NOT club-3090#1195 — that is an intermittent 500 parsing the model's OUTPUT. Same subsystem, opposite end. The caveat says so in all 18 composes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the tool path on all 18 GLM composes, and corrects a mechanism I got wrong in
#1254.
From #1250 (@paulp83), and reported
independently upstream on ggml-org/llama.cpp#27754
on 2026-09-07 where it got no reply.
The line in the error is the crash site, not the cause
Two earlier readings were wrong. Both are recorded in the patch README so nobody re-derives them:
.items()" —GLM-4.7-Flash.jinjacarries the identicalconstruct, is exercised by llama.cpp's own autoparser tests, and passes.
supports_object_argumentsbecause the template iteratesrather than indexing" — adding a named access changes nothing; the probe never gets that far.
Actual cause: minja does not implement numeric dotted attribute access (
x.0). Jinja2 definesx.0asx[0]; minja parses it as a non-computed member whose property is a number literal, andthrows. The embedded template uses it in 4 places, so the caps probe throws, infers nothing, and
leaves every capability false — including
supports_object_arguments, which gates the JSON-string →object conversion of tool arguments in
chat.cpp. Arguments stay a string, and then theautoparser dies on
.items()at line 163.The tell that the earlier readings were wrong:
supports_string_contentis also false, which notools-only explanation accounts for.
The patch — 5 lines, wire format untouched
Tool calls still render
<tool_call>NAME<arg_key>K</arg_key><arg_value>V</arg_value>…. That isload-bearing: change the format and the generated parser stops matching what the model emits.
Verified without loading 140 GiB, with llama.cpp's own tooling
llama-template-analysis --template-file <t>:supports_toolssupports_tool_callssupports_parallel_tool_callssupports_string_contentllama-debug-template-parser(the real autoparser path) on the override yieldstool_mode: TAG_WITH_TAGGED,per_call_start '<tool_call>', and a full PEG parser + GBNF carryingthe
<arg_key>/<arg_value>rules.Reference Jinja2 3.1.2: original ≡ override byte-identical across 8 cases, including the
two-call sort path,
reasoning_content, typed content, list-of-outputs tool content, and a<think>-split message.No live boot. GLM-5.3-Flash is ~140 GiB and this has not been served end-to-end. Everything
above is static analysis plus the engine's own autoparser.
The production pin is
server-cuda-b10236; the analysis build came from a local checkout whoseminja may differ. The error text and byte offset match exactly, so the mechanism holds, but the two
builds are not proven identical.
Drift guard asserts positively
A re-vendored vendor template without the fix fails silently — boot green,
/healthgreen, toolcalls 400. So the guard requires the caps to read true rather than merely checking the
.0.string is absent, and pairs that with a tool-call smoke and a 3-turn loop.
Upstream
Not fixed upstream as of 2026-09-11. Right fix is in minja: implement
x.0asx[0], or have thecaps probe degrade gracefully rather than inferring nothing when a probe throws. Closest prior art
is #28509 (Gemma 4 misclassified
supports_typed_content), same subsystem, closed completed. An issue is drafted for the maintainerto post — ggml-org bars autonomous agent contributions.