[4/4] fix(data): align thinking-model chat templates and masks

[yaoyu-33]

Summary

• retain the existing offline Qwen 3.5/3.6 tools × thinking coverage
• translate canonical thinking controls to the native GLM-5.2 and Kimi-K3 tokenizer arguments
• infer assistant-only loss boundaries for GLM-5.2, Kimi-K3, and MiniMax-M3 templates that do not expose HF generation masks
• preserve OpenAI JSON-string tool arguments, tools, right truncation, padding, and shifted-label alignment
• keep MiniMax thinking_mode="adaptive" unchanged

This PR is stacked on #5538.

Fixes

• GLM-5.2: map truncate_history_thinking to native clear_thinking; use conversation-prefix renders because completed assistant turns have no explicit end token
• Kimi-K3: map enable_thinking to native thinking; detect the custom XTML assistant structure without depending on the remote-code class name
• MiniMax-M3: recognize its assistant/tool role markers and include the assistant turn terminator in loss
• shared preprocessing: preserve chat_template_kwargs and top-level tools when re-rendering a conversation to derive its mask

The review also removed unconditional signature/vocabulary inspection from the per-row path. Vocabulary probing now occurs only for template-less tokenizers that expose the Kimi-style thinking parameter.

Official template audit

Pinned snapshots checked on 2026-08-13:

Model	Revision	Cases	MBridge ↔ native serving text/IDs
zai-org/GLM-5.2	b4734de4facf877f85769a911abafc5283eab3d9	4	exact / exact
moonshotai/Kimi-K3	9f62e4e9fffbd0a83ddd60e1c209d828994b3569	4	exact / exact
MiniMaxAI/MiniMax-M3	f0e1c1e04d40177e4673a22097036854f536e9c0	4	exact / exact

The four cases for each model are with/without tools × with/without thinking. Raw OpenAI JSON-string function.arguments is used in tool cases.

The existing Qwen audit remains exact in all 8 Qwen 3.5/3.6 cases.

Loss-mask audit

MBridge masks system/user/tool-response content and assistant role headers. It supervises retained thinking, assistant responses, assistant tool calls, and model-defined assistant turn terminators where present.

Against ms-swift ca937fbaf8e0c3dc4ea34358889430e36475463b:

• Kimi-K3 thinking cases: input IDs and raw assistant mask exact, with and without tools
• MiniMax-M3 non-thinking cases: input IDs and raw assistant mask exact, with and without tools
• GLM-5.2: render differs because ms-swift appends the next role sentinel to completed assistant rows
• Kimi-K3 non-thinking: render differs because ms-swift inserts an empty think channel while the official tokenizer omits it
• MiniMax-M3 thinking: render differs because ms-swift prepends an extra closing think marker

Rows with different input IDs are reported as render differences rather than misleading index-level mask differences.

Tests

• isolated CPU pytest: 28 passed across GLM/Kimi/MiniMax, Qwen 3.5/3.6, and Nemotron Lightning chat suites
• GLM/Kimi/MiniMax semantic matrix: 16 passed, including truncation/padding and MiniMax adaptive mode
• pinned official tokenizer audit: rendered text/input IDs exact in 12/12 new cases
• uv run --no-sync pre-commit run --all-files
• uv run --no-sync python -m compileall -q src/megatron/bridge/data/conversation_processing.py tests/unit_tests/models/test_glm52_kimi_k3_minimax_m3_chat.py

A normal project-environment pytest startup is not available on the local manylinux 2.31 host because the pinned nvidia-resiliency-ext wheel requires manylinux 2.39; CI provides the authoritative project-environment run.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[yaoyu-33]

/ok to test 6f3db7f

[claude]

Light review - LGTM

Docs + unit-test-only PR. Verified the new test against the source it exercises:

• resolve_chat_template_kwargs (conversation_processing.py:489) translates truncate_history_thinking to preserve_thinking only when the template exposes preserve_thinking; the 3.5 vs 3.6 fake tokenizers (differing chat templates) correctly probe both branches.
• _normalize_assistant_tool_call_arguments (conversation_processing.py:532) parses the OpenAI JSON-string function.arguments into a dict; the test asserts the Seattle args are normalized -- matches.
• text_chat_collate_fn / tokenize_chat_example signatures and TokenizedConversation.assistant_mask line up with the assertions.
• README additions accurately describe the masking/supervision boundaries and the preserve_thinking translation.

No critical bugs, typos, or doc inaccuracies found. Could not execute the test in this sandbox, but its assumptions are consistent with the source paths.

Suggested test cases

• No perf tests impacted.

[yaoyu-33]

/ok to test 49a164c

[yaoyu-33]

/ok to test 14cadec

[yaoyu-33]

/ok to test 14cadec

[yaoyu-33]

/ok to test 4c92760