[kimbochen]

Summary

Adds a SPEED-Bench acceptance-length collector for XiaomiMiMo/MiMo-V2.6-Flash-RL's DFlash drafter on vLLM, and commits the golden curve it measured, infx/golden_al_distribution/mimov2.6flash_dflash.yaml. AgentX needs this curve before any MiMo DFlash recipe (#4330 adds the non-speculative SGLang and vLLM keys) can launch.

What it measures: benchmarks/single_node/speedbench/mimov2.6flash_fp4_b300_vllm.sh, dispatched through speedbench-al.yml with framework=vllm.

• The drafter is the checkpoint's own dflash/ subfolder (5 layers, BF16), served as shipped through --speculative-config '{"method": "dflash", ...}' with greedy drafting. This is the speculative path in vLLM's MiMo-V2.6-Flash-RL recipe.
• Draft lengths 1–7, because the drafter's block_size is 8. The script rejects any other value before starting a server.
• Thinking off and on, through --chat-template-kwargs (enable_thinking: false / true). The script refuses thinking-on kwargs that do not set enable_thinking: true, since the workflow default targets other models.
• Server: vllm/vllm-openai:v0.31.0, TP4, mimo parsers, FULL_DECODE_ONLY CUDA graphs as in vLLM's DFlash recipe, prefix caching off.
• Sampling: temperature 1.0 and top-p 0.95, from the model card. Concurrency 32.
• AL = 1 + Δaccepted / Δdrafts from vLLM's spec-decode counters around each pass.
• The client runs through vLLM's Python benchmark entrypoint, because newer vLLM hands vllm bench serve to a Rust client that lacks the SPEED-Bench flags. Those flags are checked once before any server starts, and a run where no cell yields an AL fails instead of passing with an empty table. All three guards match the MiniMax-M3.1-Flash collector.
• If port 8888 is already taken on the node, the script logs the holder and serves on a free port. Run 41345416194 lost every cell to Address already in use on dsxe-sa-b300-prd0-gpu-00, where a process outside the Slurm job holds 8888.

Why DFlash rather than the native MTP layers: vLLM v0.31.0 loads only the first of the checkpoint's 3 MTP layers and reuses it at every draft step, so an MTP curve would understate the model. DFlash is the drafter vLLM documents for this checkpoint.

No perf-changelog entry: no benchmark point changes.

Golden curve: from run 41346734342 on B300. All 14 cells completed 80/80 requests with no failures or empty outputs. Every thinking-on output carries reasoning tags and no thinking-off output does, so the mode switch took effect.

Draft tokens	1	2	3	4	5	6	7
thinking_on	1.84	2.51	2.97	3.35	3.76	3.86	4.08
thinking_off	1.83	2.52	2.95	3.30	3.74	3.89	4.00

The curve is registered in both golden-curve READMEs and in the curve-name test. It sits close to the other DFlash-family curve, Muse Spark Open DFlash2 (thinking-on 1.85 to 4.05).

To re-collect, dispatch from this branch:

gh workflow run speedbench-al.yml --repo SemiAnalysisAI/InferenceX --ref feat/mimov2.6flash-speedbench-collector \
  -f runner=b300 -f framework=vllm \
  -f model=XiaomiMiMo/MiMo-V2.6-Flash-RL -f model-prefix=mimov2.6flash \
  -f image='vllm/vllm-openai:v0.31.0@sha256:c1c9f6fd5c109ba7f0546a59f5b2f15fb87f64c77782e90a27b648b42a8e67c3' \
  -f 'mtp-list=1 2 3 4 5 6 7' -f 'thinking-modes=off on' \
  -f thinking-kwargs='{"enable_thinking": true}' \
  -f category=coding -f output-len=4096 -f open-pr=false

The checkpoint, including dflash/, is now staged in b300-dsxe's writable model root.

AI model disclosure

• Model/version: Claude Opus 5.5 (claude-opus-5-5), via Claude Code. One Claude Haiku subagent extracted lines from a CI log.
• Role: SGLang and vLLM source review, collector authoring, local validation, CI monitoring and fixes, and PR preparation.

PR checklist and change type

Type of Change

• Bug fix
• New feature
• Configuration change
• Documentation update
• Other (please describe)

Checklist

• I have completed the AI model disclosure and kept it current
• I have tested my changes locally
• I have updated documentation if necessary
• If python -m infx.matrix.changed --base origin/main lists affected config keys, I have appended one new entry covering them to the physical end of inferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entries
• If this PR appends an inferencex-e2e/perf-changelog.yaml entry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs): full-sweep-fail-fast (recommended), full-sweep-enabled, or non-canary-full-sweep-enabled. Optional modifiers all-evals, evals-only, and agentx-fast require a primary label; the last two block reuse while applied.
• Before merging via reuse, an authorized maintainer (OWNER/MEMBER/COLLABORATOR) has commented /use <run_id> (or the legacy /reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.

中文

为 XiaomiMiMo/MiMo-V2.6-Flash-RL 自带的 DFlash 草稿模型新增基于 vLLM 的 SPEED-Bench 接受长度（AL）采集脚本，并提交实测得到的 golden 曲线 infx/golden_al_distribution/mimov2.6flash_dflash.yaml。MiMo 的 DFlash AgentX 配方必须先有这条曲线才能启动（#4330 新增了不带投机解码的 SGLang 与 vLLM 配置）。

测量内容： benchmarks/single_node/speedbench/mimov2.6flash_fp4_b300_vllm.sh，通过 speedbench-al.yml（framework=vllm）调度。

• 草稿模型为检查点自带的 dflash/ 子目录（5 层，BF16），按原样通过 --speculative-config '{"method": "dflash", ...}' 加载，采用贪心草稿采样。这正是 vLLM 的 MiMo-V2.6-Flash-RL 配方 中的投机解码方式。
• 草稿长度为 1–7，因为草稿模型的 block_size 为 8。脚本会在启动服务前拒绝其他值。
• 分别关闭和开启思考，通过 --chat-template-kwargs 传入（enable_thinking: false / true）。若开启思考的 kwargs 未设置 enable_thinking: true，脚本会拒绝运行，因为工作流的默认值面向其他模型。
• 服务：vllm/vllm-openai:v0.31.0、TP4、mimo 解析器、与 vLLM DFlash 配方一致的 FULL_DECODE_ONLY CUDA graph，关闭前缀缓存。
• 采样：temperature 1.0、top-p 0.95（来自模型卡）。并发 32。
• AL = 1 + Δ接受 token 数 / Δ草稿次数，取自每轮前后 vLLM 的投机解码计数器。
• 客户端走 vLLM 的 Python 基准入口，因为新版 vLLM 会把 vllm bench serve 交给缺少 SPEED-Bench 参数的 Rust 客户端。这些参数会在启动任何服务前预检一次；若没有任何单元得到 AL，作业会失败，而不是以空表通过。三项防护均与 MiniMax-M3.1-Flash 采集脚本一致。
• 若节点上的 8888 端口已被占用，脚本会记录占用者并改用空闲端口。运行 41345416194 在 dsxe-sa-b300-prd0-gpu-00 上所有单元都因 Address already in use 失败，该节点上有 Slurm 作业之外的进程占用 8888。

为何选 DFlash 而非自带 MTP 层： vLLM v0.31.0 只加载检查点 3 层 MTP 中的第一层，并在每个草稿步骤重复使用，因此 MTP 曲线会低估模型能力。DFlash 是 vLLM 为该检查点给出的草稿模型。

无需 perf-changelog 条目： 不改变任何基准测试点。

黄金曲线： 来自 B300 上的运行 41346734342。14 个单元均完成 80/80 请求，无失败、无空输出。thinking-on 输出全部带推理标签，thinking-off 输出均不带，说明思考开关生效。

草稿 token 数	1	2	3	4	5	6	7
thinking_on	1.84	2.51	2.97	3.35	3.76	3.86	4.08
thinking_off	1.83	2.52	2.95	3.30	3.74	3.89	4.00

该曲线已登记在中英文 golden 曲线 README 与曲线名测试中，与另一条 DFlash 系曲线 Muse Spark Open DFlash2（thinking-on 1.85 至 4.05）接近。如需重新采集，可用上方英文部分的命令直接从本分支调度。检查点（含 dflash/）现已存放在 b300-dsxe 的可写模型目录中。

AI 模型使用说明： Claude Opus 5.5（claude-opus-5-5），通过 Claude Code 使用，另有一个 Claude Haiku 子 agent 从 CI 日志中提取行；负责 SGLang 与 vLLM 源码审阅、采集脚本编写、本地验证、CI 监控与修复以及 PR 准备。

改动类型： 新功能。

🤖 Generated with Claude Code

https://claude.ai/code/session_01BGggnGqhy9dML7NM9ow3Fm

[github-actions]

Thanks for the contribution!

• Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
• PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
• After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.

中文

感谢你的贡献！

• **审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
• **PR 验证：**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目，并且恰好带有一个主标签时，才会运行扫描：full-sweep-fail-fast（强烈推荐；canary 加逐矩阵 fail-fast）、full-sweep-enabled（有 canary；矩阵任务在失败后继续运行）或 non-canary-full-sweep-enabled（无 canary，也无 fail-fast）。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
• **合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

[claude]

Findings marked 🟡 are optional suggestions and need no follow-up push.

[claude]

🔴 AgentX may get a golden curve labelled thinking_on that was measured with thinking off, if MiMo's chat template defaults enable_thinking to false. The script never checks the mode it serves. It writes every cell under thinking_on: (line 219), on the unchecked assumption at line 19 that the template default is thinking on. Fix: before writing thinking_on, confirm that the rendered prompt opens a thinking block, or that the model's chat_template defaults enable_thinking to true. Fail with CRITICAL if not, as is already done for THINKING_MODES.

Why this was flagged

The trigger is MiMo-V2.6-Flash-RL's chat_template leaving thinking off unless enable_thinking is passed. Earlier MiMo-V2 model cards enable thinking through chat_template_kwargs, so this case is realistic, and it cannot be checked from this checkout. The PR says SGLang's speed-bench loader applies the template client-side with no kwargs, so each cell (run_cell at lines 136-203) uses the template's default mode. The THINKING_MODES guard at line 33 only checks the caller's string, never what the model produces. The YAML block at lines 209-223 then writes the measured AL under thinking_on:, and the header at line 212 says "thinking_on: chat template default". That file becomes infx/golden_al_distribution/mimov2.6flash_mtp.yaml, which AgentX reads for thinking_on. Non-thinking AL usually differs from reasoning-trace AL, so synthetic acceptance in MiMo speculative recipes would be wrong and nothing would flag it. No golden curve exists on the base branch, so nothing is mislabelled there.

Verification: The guard at lines 27-30 (if [[ "$THINKING_MODES" != "on" ]]) only checks the caller's string. No chat_template_kwargs or enable_thinking is passed anywhere. The YAML block writes echo " thinking_on:" (line ~219) without checking anything. If the template defaults to non-thinking, the golden curve is recorded under thinking_on but measured with thinking off.

[claude]

🟡 (optional) If the first MiMo checkpoint download is interrupted, every later collection run reuses the partial checkpoint and fails, and no run ever retries the download. Lines 47-55 run hf download only when $MODEL_PATH is missing or empty. They also ignore its exit status. Once hf download starts, its .cache/huggingface metadata makes the directory non-empty, so later runs skip the download. Fix: when the checkpoint is not staged, always run hf download (it resumes and skips finished files) and exit when it fails. That way a partial checkpoint is never served.

Why this was flagged

MiMo-V2.6-Flash-RL is not in the b300-dsxe models.entries (configs/runners.yaml:515-535). resolve_model (infx/launch/drivers/script.py:47-53) therefore points MODEL_PATH at writable-models/, and the guard at mimov2.6flash_fp4_b300_sglang.sh:47 is the only thing that fills it. A first run whose hf download (line 54) stops early leaves files behind. Its exit status is never checked, and the script has no set -e. Once hf download --local-dir starts, it writes .cache/huggingface/download/ inside the directory. From then on ls -A is non-empty and every later run skips the download. If config.json is missing, the python at lines 57-60 fails and MTP_LAYERS is empty. If config.json is present but shards are missing, each sglang.launch_server fails. Either way the state persists until someone deletes /data/home/sa-gha-runner/models/ by hand.

Verification: The trigger is a first-run hf download that stops partway while it fills the persistent writable-models directory. Line 54 ignores its exit status, and line 21 is only set -o pipefail, with no -e. After an interrupted run ls -A is non-empty and every later run skips the download. Every run then fails until someone deletes the directory by hand.

[claude]

🟡 (optional) Maintainers get a new script that opens with a 17-line descriptive header, which AGENTS.md's mandatory comment rules forbid. Lines 3-19 describe what the file does, how SGLang serves MiMoV2 MTP, what AL means, and how to dispatch the script. Fix: delete the header block. Move any technical reason that a line needs, such as why --speculative-num-draft-tokens is mtp + 1 or why only thinking_on is accepted, into one short comment on that line.

Why this was flagged

The diff adds the new file inferencex-e2e/benchmarks/single_node/speedbench/mimov2.6flash_fp4_b300_sglang.sh with a block comment at lines 3-19. That block summarises the collector, the SGLang MTP mapping, the AL definition and the dispatch path. AGENTS.md lines 60-66 ("Code comments (mandatory)") say "Never add broad, overarching comments at the top of a file" and "Do not describe what the file, topology, or config is". Comments may only explain a specific line. The sibling musespark_fp4_b300_sglang.sh carries the same kind of header. That shows earlier drift, not permission, and the rule applies to every new file. Without a fix, the merged file breaks the repo's mandatory convention and adds descriptive prose that will go stale as the flags change.

Verification: Lines 3-19 of inferencex-e2e/benchmarks/single_node/speedbench/mimov2.6flash_fp4_b300_sglang.sh are a 17-line comment block placed before any code. AGENTS.md line 64 says: "No header comments. Do not describe what the file, topology, or config is." This block does what the rule forbids. Nothing breaks at runtime; the only effect is that a repository rule is broken.

[github-actions]

inferencex-e2e/infx diff: +30 -0

Generally most InferenceX submissions don't need to add changes to the infx package. On the off chance they do, generally the diff is small. Please prompt your agent to clean your PR to keep your diff to only necessary & minimal changes too. We have an increased code quality bar for changes to the infx package.

中文

大多数 InferenceX 提交通常不需要修改 infx 包。即使确实需要，改动通常也很小。请让你的 agent 清理 PR，使改动只包含必要且最小的内容。infx 包的改动有更高的代码质量要求。