[kimbochen]

Summary

Adds AgentX coverage for XiaomiMiMo/MiMo-V2.6-Flash-RL, a 309B-total / 15B-active MiMoV2 model, on B200 with upstream SGLang and upstream vLLM. Neither key uses speculative decoding yet.

What runs: two new keys, 24 points in total. Every point keeps the KV cache on the GPU (no CPU offload).

Key	Image	Points
mimov2.6flash-fp4-b200-sglang-agentic	lmsysorg/sglang:v0.5.21-cu130	TP4 at conc 1, 4, 8, 16, 32, 64; TP2 at 1, 4, 8, 16, 32
mimov2.6flash-fp4-b200-vllm-agentic	vllm/vllm-openai:v0.31.0	TP8 at conc 1, 4; TP4 at 1, 4, 8, 16, 32, 64; TP2 at 1, 4, 8, 16, 32

Both images are pinned by digest. Admission is 2× concurrency for subagent fan-out, and decode CUDA graphs cover the concurrency.

Precision (fp4), checked against the checkpoint's safetensors headers:

• Routed experts, about 98% of the parameters, are packed MXFP4 (E2M1 with an E8M0 scale per 32 values).
• Dense MLP, fused QKV and MTP layers are FP8 E4M3 with 128×128 block scales.
• o_proj, router, embeddings and lm_head are BF16.
• On B200 both engines run the routed experts natively as MXFP4 weights × MXFP8 activations (FlashInfer TRT-LLM), and the FP8 layers as block-scaled FP8. This is the same mixed layout as deepseek-ai/DeepSeek-V4.1-Flash, which this repo labels fp4.

SGLang settings (from the SGLang MiMo-V2.5 cookbook; V2.6-Flash uses the same MiMoV2ForCausalLM architecture and size class):

• moe-runner-backend: flashinfer_mxfp4. v0.5.21's MiMo default picks the FP8 TRT-LLM runner on SM100 for any quant_method: fp8 checkpoint, which cannot read packed MXFP4 experts. The upstream fix ([Fix] Select the MXFP4 MoE runner for MiMo-V2 packed experts on SM100 sgl-project/sglang#41668) landed after v0.5.21.
• attention-backend: fa4, because the default trtllm_mha rejects MiMoV2's asymmetric K/V head dims on Blackwell.
• No language-model-only: v0.5.21 rejects it for MiMoV2ForCausalLM (the first canary failed at startup on it). The vision and audio encoders (~2 GB) therefore load, with mm-attention-backend: fa4 for the ViT; text-only AgentX requests never run them.
• chunked-prefill-size 16384, mem-fraction-static 0.85, mimo reasoning and tool-call parsers.
• No TP8: the fused QKV weights are interleaved for 4-way TP (num_key_value_heads = 4), and SGLang requires the attention TP size to divide 4.

vLLM settings (from vLLM's MiMo-V2.6-Flash-RL recipe, which names v0.31.0 as the first release that loads this checkpoint):

• gpu-memory-utilization 0.95, max-model-len auto (the largest context the KV pool holds, for the uncapped AgentX corpus), max-num-batched-tokens 16384, mimo parsers.
• TP8 is valid here: vLLM regroups the fused QKV per rank and replicates KV heads.
• vLLM's MiMoV2ForCausalLM is text-only, so no language-model-only flag is needed.
• KV cache stays BF16: the MiMo FP8 KV fix ([Attention][Model] FP8 KV cache for Triton DiffKV and MiMo-V2.6-Flash vllm-project/vllm#58128) landed after v0.31.0.

Thinking stays on for both engines, which is the chat template's default.

Known gaps:

• No speculative decoding yet. infx/golden_al_distribution/ has no MiMo curve, so a speculative run would fail at launch. feat(speedbench): add MiMo-V2.6-Flash vLLM DFlash AL collector and golden curve / feat(speedbench)：新增 MiMo-V2.6-Flash vLLM DFlash AL 采集脚本与黄金曲线 #4331 adds a vLLM collector for the checkpoint's DFlash drafter; DFlash arms follow once that curve is committed.
• Weights are not pre-staged on b200-nscale. runners.yaml has no entry for this model, so hf:XiaomiMiMo/MiMo-V2.6-Flash-RL is served from the shared HF hub cache, the same way as DeepSeek-V4.1-Flash on this cluster. The repo is not gated. The first job downloads the checkpoint.
• Checkpoint choice. Xiaomi has since released XiaomiMiMo/MiMo-V2.6-Flash-MOPD (same architecture and format), which fixes the RL release's tool-call repetition. This PR stays on the RL checkpoint; MOPD can be added separately.
• Not in the engine-mapping table. This PR adds only the MODELS.md support-matrix row; maintainers decide the model-to-engine mapping.
• Not yet run end to end on GPUs. The first canary failed at server startup on language-model-only (fixed above); the full-sweep-fail-fast sweep reruns on the latest commit.

AI model disclosure

• Model/version: Claude Opus 5.5 (claude-opus-5-5), via Claude Code. No delegated agents.
• Role: research, recipe and config authoring, local validation, sweep debugging, and PR preparation.

PR checklist and change type

Type of Change

• Bug fix
• New feature
• Configuration change
• Documentation update
• Other (please describe)

Checklist

• I have completed the AI model disclosure and kept it current
• I have tested my changes locally
• I have updated documentation if necessary
• If python -m infx.matrix.changed --base origin/main lists affected config keys, I have appended one new entry covering them to the physical end of inferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entries
• If this PR appends an inferencex-e2e/perf-changelog.yaml entry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs): full-sweep-fail-fast (recommended), full-sweep-enabled, or non-canary-full-sweep-enabled. Optional modifiers all-evals, evals-only, and agentx-fast require a primary label; the last two block reuse while applied.
• Before merging via reuse, an authorized maintainer (OWNER/MEMBER/COLLABORATOR) has commented /use <run_id> (or the legacy /reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.

中文

为 XiaomiMiMo/MiMo-V2.6-Flash-RL（MiMoV2 架构，总参数 309B / 激活 15B）在 B200 上新增基于上游 SGLang 与上游 vLLM 的 AgentX 覆盖。两个配置键暂不使用投机解码。

运行内容： 新增两个配置键，共 24 个点。所有点的 KV 缓存都保留在 GPU 上（不做 CPU 卸载）。

配置键	镜像	测试点
mimov2.6flash-fp4-b200-sglang-agentic	lmsysorg/sglang:v0.5.21-cu130	TP4，并发 1、4、8、16、32、64；TP2，并发 1、4、8、16、32
mimov2.6flash-fp4-b200-vllm-agentic	vllm/vllm-openai:v0.31.0	TP8，并发 1、4；TP4，并发 1、4、8、16、32、64；TP2，并发 1、4、8、16、32

两个镜像均按 digest 固定。准入上限为并发的 2 倍，以容纳子 agent 扇出；decode CUDA graph 覆盖到并发数。

精度（fp4），已对照检查点的 safetensors 头信息核实：

• 路由专家约占参数量的 98%，以打包的 MXFP4 存储（E2M1，每 32 个值一个 E8M0 缩放因子）。
• 稠密 MLP、融合 QKV 与 MTP 层为 FP8 E4M3，采用 128×128 分块缩放。
• o_proj、路由器、词嵌入和 lm_head 为 BF16。
• 在 B200 上，两个引擎都以原生 MXFP4 权重 × MXFP8 激活（FlashInfer TRT-LLM）运行路由专家，FP8 层按分块缩放 FP8 运行。这与 deepseek-ai/DeepSeek-V4.1-Flash 的混合布局相同，本仓库将其标记为 fp4。

SGLang 设置（参照 SGLang MiMo-V2.5 cookbook；V2.6-Flash 与 V2.5 采用相同的 MiMoV2ForCausalLM 架构，规模也相同）：

• moe-runner-backend: flashinfer_mxfp4。v0.5.21 的 MiMo 默认配置会在 SM100 上为任何 quant_method: fp8 的检查点选择 FP8 TRT-LLM runner，而它无法读取打包的 MXFP4 专家。上游修复（[Fix] Select the MXFP4 MoE runner for MiMo-V2 packed experts on SM100 sgl-project/sglang#41668）在 v0.5.21 之后才合入。
• attention-backend: fa4：在 Blackwell 上，默认的 trtllm_mha 不支持 MiMoV2 的非对称 K/V head 维度。
• 不使用 language-model-only：v0.5.21 对 MiMoV2ForCausalLM 会拒绝该参数（首次 canary 因此在启动时失败）。因此会加载视觉与音频编码器（约 2 GB），ViT 使用 mm-attention-backend: fa4；纯文本的 AgentX 请求不会调用它们。
• chunked-prefill-size 16384、mem-fraction-static 0.85、mimo 推理与工具调用解析器。
• 无 TP8：融合 QKV 权重按 4 路 TP 交错存储（num_key_value_heads = 4），SGLang 要求注意力 TP 大小能整除 4。

vLLM 设置（参照 vLLM 的 MiMo-V2.6-Flash-RL 配方，其中指出 v0.31.0 是首个能加载该检查点的版本）：

• gpu-memory-utilization 0.95、max-model-len auto（KV 池能容纳的最大上下文，适配不截断的 AgentX 轨迹）、max-num-batched-tokens 16384、mimo 解析器。
• vLLM 可以跑 TP8：它会按 rank 重新组织融合 QKV，并复制 KV head。
• vLLM 的 MiMoV2ForCausalLM 只含文本部分，因此无需 language-model-only。
• KV 缓存保持 BF16：MiMo 的 FP8 KV 修复（[Attention][Model] FP8 KV cache for Triton DiffKV and MiMo-V2.6-Flash vllm-project/vllm#58128）在 v0.31.0 之后才合入。

两个引擎都保持开启思考模式，这是 chat template 的默认行为。

已知缺口：

• 暂未启用投机解码。 infx/golden_al_distribution/ 中还没有 MiMo 曲线，投机解码运行会在启动时失败。feat(speedbench): add MiMo-V2.6-Flash vLLM DFlash AL collector and golden curve / feat(speedbench)：新增 MiMo-V2.6-Flash vLLM DFlash AL 采集脚本与黄金曲线 #4331 新增针对检查点 DFlash 草稿模型的 vLLM 采集脚本；曲线提交后再加入 DFlash 配置。
• b200-nscale 上未预置权重。 runners.yaml 中没有该模型的条目，因此 hf:XiaomiMiMo/MiMo-V2.6-Flash-RL 从共享 HF hub 缓存加载，与 DeepSeek-V4.1-Flash 在该集群上的做法相同。该仓库不需要申请访问权限，首个任务会下载检查点。
• 检查点选择。 小米随后发布了 XiaomiMiMo/MiMo-V2.6-Flash-MOPD（架构与格式相同），修复了 RL 版本的工具调用重复问题。本 PR 仍使用 RL 检查点，MOPD 可另行添加。
• 未加入引擎映射表。 本 PR 只在 MODELS.md 支持矩阵中新增一行，模型与引擎的映射由维护者决定。
• 尚未在 GPU 上完整跑通。 首次 canary 因 language-model-only 在服务启动时失败（已在上文修复）；full-sweep-fail-fast sweep 会在最新提交上重新运行。

AI 模型使用说明： Claude Opus 5.5（claude-opus-5-5），通过 Claude Code 使用，未委派其他 agent；负责调研、配方与配置编写、本地验证、sweep 排错以及 PR 准备。

改动类型： 新功能、配置变更、文档更新。

🤖 Generated with Claude Code

[claude]

Findings marked 🟡 are optional suggestions and need no follow-up push.

[claude]

🔴 Maintainers merging this PR with the supported tools get a hard refusal, because the new entry's pr-link: TBD is not a recognised placeholder. PR_LINK_PLACEHOLDERS (validate_perf_changelog.py:22) accepts only XXX and https://github.com/SemiAnalysisAI/InferenceX/pull/XXX. compare_entries rejects anything else on PR runs. Fix: set the pr-link to https://github.com/SemiAnalysisAI/InferenceX/pull/XXX, as in CONTRIBUTING.md:206, or to this PR's canonical URL, so merge_with_reuse and resolve-conflict can rewrite it at merge.

Why this was flagged

The appended entry ends with pr-link: TBD (perf-changelog.yaml:10240). merge_with_reuse is the documented merge path for reuse (.github/workflows/README.md:289). It calls canonicalize_appended_links (merge_with_reuse.py:478-483). That function raises "appended entry N has unexpected pr-link 'TBD'" when a link is not in PR_LINK_PLACEHOLDERS (prepare_perf_changelog_merge.py:105-111). Before that, compare_entries -> validate_added_pr_link raises "new PR entry must use 'https://github.com/SemiAnalysisAI/InferenceX/pull/' or an XXX placeholder; found 'TBD'" (validate_perf_changelog.py:139-150). The changelog-conflict helper resolve_conflict_bytes calls compare_entries too (prepare_perf_changelog_merge.py:146), so recovery after a main-side changelog conflict is also refused. The PR-time check in run-sweep.yml only runs infx.matrix.plan through validate_matrix_compatible_change, which never calls compare_entries. So sweeps pass and the problem shows up only at merge or conflict time.

Verification: The new entry ends with pr-link: TBD (perf-changelog.yaml:10240). PR_LINK_PLACEHOLDERS holds only "XXX" and "https://github.com/SemiAnalysisAI/InferenceX/pull/XXX" (validate_perf_changelog.py:22-25). merge_pr calls canonicalize_changelog(pr, repo) unconditionally (merge_with_reuse.py:631). The PR-time CI check (run-sweep.yml:97-117) never checks the pr-link.

[claude]

🟡 nit (optional): The new recipe adds a three-line header comment above the overrides section, which the repo's comment rules forbid. "One variant per point" describes the file layout. The TP8 note explains a variant that is absent and is not tied to any line. Fix: delete the block. If the 2x CONC admission needs a reason, put one short line next to the first max-running-requests. The TP8 reasoning is already in perf-changelog.yaml.

Why this was flagged

AGENTS.md "Code comments (mandatory)" applies to recipes. It says: "Never add broad, overarching comments at the top of a file or above a recipe, override, or section. Do not describe what the file, topology, or config is." The block at agentic.yaml:50-52 sits directly above override_tp4_c1. It describes the variant layout ("One variant per point") and justifies why TP8 is missing, so it explains no particular line. The nearest existing pattern, dsv41flash/sglang/b200-fp4-mtp/agentic.yaml:51, starts its overrides with no header comment. The perf-changelog entry this PR adds already records the TP8 rationale.

Verification: nit. The new recipe agentic.yaml has a three-line block at lines 50-52, directly above override_tp4_c1:. AGENTS.md:64 says: "No header comments. Never add broad, overarching comments at the top of a file or above a recipe, override, or section." The block is a section-level comment over all 11 overrides. Nothing breaks at runtime because YAML ignores comments.

[github-actions]

View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=39323832019

View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=39323832019

[kimbochen]

Dashboard support for this model is in SemiAnalysisAI/InferenceX-app#1512. It registers the mimov2.6flash DB key as the experimental MiMo-V2.6-Flash model, AgentX only. Merge or deploy #1512 before this PR's results are ingested; otherwise the rows are skipped and run 39323832019 has to be re-ingested.

中文

本模型的仪表板支持在 SemiAnalysisAI/InferenceX-app#1512 中：它将 mimov2.6flash DB 键注册为实验性模型 MiMo-V2.6-Flash，仅用于 AgentX。请先合并或部署 #1512，再导入本 PR 的结果；否则这些数据会被跳过，需要重新导入 run 39323832019。

[kimbochen]

/use 39323832019

[github-actions]

@kimbochen staging run 39323832019 failed. Inspect the staging workflow.