Skip to content

fix(glm): vendor a minja-compatible chat template — tool calling works, and the #1254 mechanism was wrong - #1255

Merged
noonghunna merged 1 commit into
masterfrom
fix/glm-minja-template-override
Sep 11, 2026
Merged

fix(glm): vendor a minja-compatible chat template — tool calling works, and the #1254 mechanism was wrong#1255
noonghunna merged 1 commit into
masterfrom
fix/glm-minja-template-override

Conversation

@noonghunna

Copy link
Copy Markdown
Owner

Fixes the tool path on all 18 GLM composes, and corrects a mechanism I got wrong in
#1254
.

From #1250 (@paulp83), and reported
independently upstream on ggml-org/llama.cpp#27754
on 2026-09-07 where it got no reply.

The line in the error is the crash site, not the cause

Two earlier readings were wrong. Both are recorded in the patch README so nobody re-derives them:

  • "minja cannot evaluate .items()"GLM-4.7-Flash.jinja carries the identical
    construct, is exercised by llama.cpp's own autoparser tests, and passes.
  • "the caps probe under-detects supports_object_arguments because the template iterates
    rather than indexing"
    — adding a named access changes nothing; the probe never gets that far.

Actual cause: minja does not implement numeric dotted attribute access (x.0). Jinja2 defines
x.0 as x[0]; minja parses it as a non-computed member whose property is a number literal, and
throws. The embedded template uses it in 4 places, so the caps probe throws, infers nothing, and
leaves every capability false — including supports_object_arguments, which gates the JSON-string →
object conversion of tool arguments in chat.cpp. Arguments stay a string, and then the
autoparser dies on .items() at line 163.

The tell that the earlier readings were wrong: supports_string_content is also false, which no
tools-only explanation accounts for.

The patch — 5 lines, wire format untouched

-{% for tool in tools %}
+{% for tool in tools if tool %}          # caps probe passes tools=[null]; 'function' in tool trips
-...m.content.0.type...        +...m.content[0].type...
-...tr.output.0.type...        +...tr.output[0].type...
-...m.content.0.output...      +...m.content[0].output...
-...entry.output.0.type...     +...entry.output[0].type...

Tool calls still render <tool_call>NAME<arg_key>K</arg_key><arg_value>V</arg_value>…. That is
load-bearing: change the format and the generated parser stops matching what the model emits.

Verified without loading 140 GiB, with llama.cpp's own tooling

llama-template-analysis --template-file <t>:

capability GGUF-embedded override
supports_tools false true
supports_tool_calls false true
supports_parallel_tool_calls false true
supports_string_content false true

llama-debug-template-parser (the real autoparser path) on the override yields
tool_mode: TAG_WITH_TAGGED, per_call_start '<tool_call>', and a full PEG parser + GBNF carrying
the <arg_key>/<arg_value> rules.

Reference Jinja2 3.1.2: original ≡ override byte-identical across 8 cases, including the
two-call sort path, reasoning_content, typed content, list-of-outputs tool content, and a
<think>-split message.

⚠️ Not verified

No live boot. GLM-5.3-Flash is ~140 GiB and this has not been served end-to-end. Everything
above is static analysis plus the engine's own autoparser.

The production pin is server-cuda-b10236; the analysis build came from a local checkout whose
minja may differ. The error text and byte offset match exactly, so the mechanism holds, but the two
builds are not proven identical.

Drift guard asserts positively

A re-vendored vendor template without the fix fails silently — boot green, /health green, tool
calls 400. So the guard requires the caps to read true rather than merely checking the .0.
string is absent, and pairs that with a tool-call smoke and a 3-turn loop.

Upstream

Not fixed upstream as of 2026-09-11. Right fix is in minja: implement x.0 as x[0], or have the
caps probe degrade gracefully rather than inferring nothing when a probe throws. Closest prior art
is #28509 (Gemma 4 misclassified
supports_typed_content), same subsystem, closed completed. An issue is drafted for the maintainer
to post — ggml-org bars autonomous agent contributions.

…lling works

Closes the tool path on all 18 GLM composes. Reported by @paulp83
(club-3090#1250) and independently upstream (ggml-org/llama.cpp#27754,
2026-09-07, unanswered).

⛔⛔ THE LINE IN THE ERROR IS THE CRASH SITE, NOT THE CAUSE. Two earlier
readings of this bug were wrong — including one I committed in #1254 — and both
are recorded in the patch README so nobody re-derives them:

  ❌ "minja cannot evaluate .items()" — GLM-4.7-Flash.jinja carries the
     identical construct, is exercised by llama.cpp's own autoparser tests, and
     passes.
  ❌ "the caps probe under-detects supports_object_arguments because the
     template iterates instead of indexing" — adding a named access changes
     nothing; the probe never gets that far.

ACTUAL CAUSE: minja does not implement NUMERIC DOTTED ATTRIBUTE ACCESS (`x.0`).
Jinja2 defines `x.0` as `x[0]`; minja parses it as a non-computed member whose
property is a number literal, and throws. The GGUF's embedded template uses it
in 4 places, so the jinja caps probe throws, infers NOTHING, and leaves EVERY
capability false — including supports_object_arguments, which gates the
JSON-string -> object conversion of tool arguments in chat.cpp. Arguments stay a
string, and the autoparser then dies on `.items()` at line 163.

That also explains supports_string_content:false, which no tools-only account
covers and which was the clue that the earlier readings were wrong.

The override is the vendor template with `.0.` -> `[0].` in 4 places plus a
null-tool guard on the tools loop (the caps probe passes tools=[null], which
trips `'function' in tool`). 5 lines. WIRE FORMAT UNCHANGED — tool calls still
render <tool_call>NAME<arg_key>K</arg_key><arg_value>V</arg_value>…, which
matters because changing it would stop the generated parser matching what the
model emits.

Verified without loading 140 GiB, using llama.cpp's own tooling:

  llama-template-analysis --template-file <template>
    embedded : tools/tool_calls/parallel/string_content  ALL FALSE
    override : ALL TRUE

  llama-debug-template-parser (the real autoparser path): override yields
    tool_mode TAG_WITH_TAGGED, per_call_start '<tool_call>', full PEG + GBNF
    with the <arg_key>/<arg_value> rules.

  Reference Jinja2 3.1.2: original ≡ override byte-identical across 8 cases
  (two-call sort path, reasoning_content, typed content, list-of-outputs tool
  content, <think> split).

⚠️ NOT live-booted. Static analysis + the engine's own autoparser, not a 140 GiB
end-to-end serve. The drift guard asserts positively (caps must read true) rather
than by absence, because a re-vendored template without the fix fails SILENTLY:
boot green, /health green, tool calls 400.

⚠️ Still NOT club-3090#1195 — that is an intermittent 500 parsing the model's
OUTPUT. Same subsystem, opposite end. The caveat says so in all 18 composes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@noonghunna
noonghunna merged commit 680fa78 into master Sep 11, 2026
@noonghunna
noonghunna deleted the fix/glm-minja-template-override branch September 11, 2026 21:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants