Repository navigation
[Klaud Cold] Add MiMo-V2.6-Flash B200 vLLM DFlash AgentX recipe / [Klaud Cold] 新增 MiMo-V2.6-Flash B200 vLLM DFlash AgentX 配方 - #4742
Conversation
…speedbench):新增 MiMo-V2.6-Flash SGLang MTP AL 采集脚本 Measures SPEED-Bench coding acceptance length for MiMo-V2.6-Flash's bundled MTP layers (SGLang multi-layer EAGLE, topk 1, depths 1-3) on B300, thinking on, for the golden AL curve mimov2.6flash_mtp.yaml. 在 B300 上测量 MiMo-V2.6-Flash 自带 MTP 层(SGLang 多层 EAGLE, topk 1,深度 1-3)在 SPEED-Bench coding 类别上的接受长度(开启思考), 用于生成 golden AL 曲线 mimov2.6flash_mtp.yaml。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…/ feat(speedbench):MiMo-V2.6-Flash AL 采集改用 vLLM DFlash Replaces the SGLang MTP collector with a vLLM v0.31.0 collector for the checkpoint's BF16 dflash/ drafter: draft lengths 1-7, thinking off and on via enable_thinking, for the golden curve mimov2.6flash_dflash.yaml. 用 vLLM v0.31.0 采集脚本替换 SGLang MTP 采集脚本,测量检查点自带的 BF16 dflash/ 草稿模型:草稿长度 1-7,通过 enable_thinking 分别关闭和 开启思考,用于 golden 曲线 mimov2.6flash_dflash.yaml。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eat/mimov2.6flash-speedbench-collector Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BGggnGqhy9dML7NM9ow3Fm
…peedbench):加固 MiMo-V2.6-Flash vLLM AL 采集脚本 Run the benchmark client through vLLM's Python entrypoint so the Rust client delegation cannot reject --speed-bench-output-len, --save-detailed or --chat-template-kwargs, preflight those flags before any server start, log a failing client exit per cell, and fail the job when no cell yields an AL value. Mirrors the MiniMax-M3.1-Flash collector. 中文:基准客户端改走 vLLM 的 Python 入口,避免 Rust 客户端拒绝 --speed-bench-output-len、--save-detailed 与 --chat-template-kwargs; 在启动任何服务前预检这些参数,逐单元记录客户端异常退出,并在没有任何 单元得到 AL 值时让作业失败。做法与 MiniMax-M3.1-Flash 采集脚本一致。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BGggnGqhy9dML7NM9ow3Fm
… is taken / fix(speedbench):8888 被占用时 MiMo AL 采集改用空闲端口 Run 41345416194 lost every cell to "Address already in use" on dsxe-sa-b300-prd0-gpu-00: a process outside the Slurm job held port 8888 in the shared network namespace. Report the holder and fall back to a free port before the first server start. 中文:运行 41345416194 在 dsxe-sa-b300-prd0-gpu-00 上所有单元都因 "Address already in use" 失败:Slurm 作业之外的进程在共享网络命名空间中 占用了 8888 端口。现在在首次启动服务前报告占用者,并改用空闲端口。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BGggnGqhy9dML7NM9ow3Fm
…t(golden-al):新增 MiMo-V2.6-Flash DFlash 黄金 AL 曲线 Commit infx/golden_al_distribution/mimov2.6flash_dflash.yaml from SPEED-Bench coding run 41346734342 (B300, vLLM v0.31.0, BF16 dflash/ drafter, TP4, temperature 1.0, top_p 0.95, output_len 4096). Every one of the 14 cells completed 80/80 requests with no failures; thinking-on outputs all carry reasoning tags and thinking-off outputs none. Register the curve in both READMEs and the curve-name test. 中文:根据 SPEED-Bench coding 运行 41346734342(B300、vLLM v0.31.0、 BF16 dflash/ 草稿模型、TP4、temperature 1.0、top_p 0.95、output_len 4096) 提交 infx/golden_al_distribution/mimov2.6flash_dflash.yaml。14 个单元均 完成 80/80 请求且无失败;thinking-on 输出全部带推理标签,thinking-off 输出均不带。同时在中英文 README 与曲线名测试中登记该曲线。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BGggnGqhy9dML7NM9ow3Fm
…aud Cold] 新增 MiMo-V2.6-Flash B200 vLLM DFlash AgentX 配方 Add mimov2.6flash-fp4-b200-vllm-agentic-dflash: upstream vLLM v0.31.0 with the checkpoint's BF16 dflash/ drafter, greedy drafting and FULL_DECODE_ONLY CUDA graphs, following vLLM's MiMo-V2.6-Flash-RL recipe. Synthetic acceptance follows the golden curve mimov2.6flash_dflash.yaml: seven draft tokens at low concurrency, three above TP4 c16 and TP2 c8. The target is pinned to revision f657ba2f, and a setup script stages that snapshot so the draft's local path exists before vLLM starts. 中文:新增 mimov2.6flash-fp4-b200-vllm-agentic-dflash:基于上游 vLLM v0.31.0,使用检查点自带的 BF16 dflash/ 草稿模型、贪心草稿采样与 FULL_DECODE_ONLY CUDA graph,遵循 vLLM 的 MiMo-V2.6-Flash-RL 配方。合成 接受长度取自黄金曲线 mimov2.6flash_dflash.yaml:低并发用 7 个草稿 token, TP4 c16、TP2 c8 以上用 3 个。目标模型固定在 f657ba2f 版本,并由 setup 脚本预先暂存该快照,确保 vLLM 启动前草稿模型的本地路径已存在。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BGggnGqhy9dML7NM9ow3Fm
|
Thanks for the contribution!
中文感谢你的贡献!
|
|
Generally most InferenceX submissions don't need to add changes to the infx package. On the off chance they do, generally the diff is small. Please prompt your agent to clean your PR to keep your diff to only necessary & minimal changes too. We have an increased code quality bar for changes to the infx package. 中文大多数 InferenceX 提交通常不需要修改 infx 包。即使确实需要,改动通常也很小。请让你的 agent 清理 PR,使改动只包含必要且最小的内容。infx 包的改动有更高的代码质量要求。 |
…Flash 变更日志条目链接到 #4742 中文:在 perf-changelog 条目中填入 PR 链接 #4742。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BGggnGqhy9dML7NM9ow3Fm
| DRAFT=/mnt/hf_hub_cache/models--XiaomiMiMo--MiMo-V2.6-Flash-RL/snapshots/$REVISION/dflash | ||
| for attempt in 1 2 3 4 5; do | ||
| if HF_HUB_CACHE=/mnt/hf_hub_cache hf download "$REPO" --revision "$REVISION" >/dev/null; then |
There was a problem hiding this comment.
🟡 (optional) The new setup hook ignores the job's HF hub cache setting and always downloads into /mnt/hf_hub_cache. If the container's HF_HUB_CACHE points elsewhere, vLLM downloads the full target a second time, with no error. Line 14 sets HF_HUB_CACHE=/mnt/hf_hub_cache unconditionally, and line 12 checks the draft at the same constant path. AGENTS.md forbids replacing a caller-supplied setting this way. Fix: validate the caller's HF_HUB_CACHE with check_env_vars and fail clearly unless it is the /mnt/hf_hub_cache path that the recipe's speculative-config names. Then download with the caller's value unchanged.
Why this was flagged
The caller sets the cache: benchmark-tmpl.yml:103 sets HF_HUB_CACHE: '/mnt/hf_hub_cache/', and infx/launch/drivers/srt/init.py:89 mounts the host cache at request.hf_hub_cache. The hook at mimov2.6flash-dflash-draft.sh:14 ignores that value and passes a constant to hf download. Line 12 builds DRAFT from the same constant. vLLM in the same container loads the target XiaomiMiMo/MiMo-V2.6-Flash-RL at revision f657ba2f… from whatever HF_HUB_CACHE the container actually has. If that differs (another dispatcher or cluster mounts the cache elsewhere, or the variable is not forwarded), the hook writes the whole checkpoint to an unmounted or different directory. vLLM then downloads it again, and nothing reports the mismatch. On the base branch this hook does not exist. The sibling kimik3-atom-dspark-draft.sh:8 relies on the forwarded HF_HUB_CACHE instead. AGENTS.md Bash (mandatory) says to "never replace a supported override with an unconditional assignment" and that receiving scripts validate configuration instead of choosing it.
Verification: nit. Trigger: the job's HF_HUB_CACHE is something other than /mnt/hf_hub_cache. In mimov2.6flash-dflash-draft.sh, line 14 runs HF_HUB_CACHE=/mnt/hf_hub_cache hf download "$REPO" --revision "$REVISION", replacing the caller's HF_HUB_CACHE with a constant. If the two paths ever differ, the failure is silent. Severity stays nit because the deployed workflows set exactly this path.
| # concrete local path, so the recipe points the draft at this snapshot's dflash/ in the | ||
| # shared hub cache. Shared-cache downloads can hit transient stale handles, hence the | ||
| # retries. | ||
| set -euo pipefail |
There was a problem hiding this comment.
🟡 (optional) The new host-setup hook turns on nounset, which the repo's mandatory Bash rules forbid in active scripts. set -euo pipefail at line 7 is listed by name in AGENTS.md under "No nounset". Fix: use set -eo pipefail. The script references no possibly-unset variables, so nothing else changes.
Why this was flagged
AGENTS.md "Bash (mandatory)" says "No nounset: no set -u, set -o nounset, set -euo pipefail, or bash -u. Keep other options such as set -eo pipefail." The new file mimov2.6flash-dflash-draft.sh:7 is an active setup script that the new recipe agentic.yaml:18 references through setup_script. Line 7 is exactly set -euo pipefail. kimik3-atom-dspark-draft.sh:5 does the same, which is where the line was copied from. That is precedent for the violation, not an exception to the rule. The PR adds a new violation of a mandatory repo rule. Dropping -u fixes it with no behaviour change.
Verification: Triggering condition: always, because the file is a new active setup script. In mimov2.6flash-dflash-draft.sh, line 7 is set -euo pipefail. Under "Bash (mandatory)", AGENTS.md:48 says: "No nounset: no set -u, set -o nounset, set -euo pipefail, or bash -u." This script is active: the new recipe agentic.yaml:18 refers to it with setup_script: mimov2.6flash-dflash-draft.sh.
| # MiMo-V2.6-Flash AgentX on B200 with vLLM and the checkpoint's own DFlash drafter | ||
| # (dflash/, BF16), following vLLM's MiMo-V2.6-Flash-RL recipe. Synthetic acceptance | ||
| # follows infx/golden_al_distribution/mimov2.6flash_dflash.yaml. |
There was a problem hiding this comment.
🟡 nit (optional): Header comments that describe a config, recipe or file break the repo's mandatory comment rule. sweep:^# (MiMo-V2.6-Flash|Stage the pinned|One variant per point) Fix: delete the 3-line header above the new master key at nvidia-master.yaml:7428-7430 and the file header at mimov2.6flash-dflash-draft.sh:2-6. Keep only short per-line technical comments, such as the existing "keep the two in sync" note.
Why this was flagged
AGENTS.md "Comments (mandatory, every file)" says "No header comments describing a file, topology, config, recipe, override, or section" and allows only short technical notes about a specific line. nvidia-master.yaml:7428-7430 adds a 3-line header describing the new mimov2.6flash-fp4-b200-vllm-agentic-dflash entry. It also cites provenance ("following vLLM's MiMo-V2.6-Flash-RL recipe"). mimov2.6flash-dflash-draft.sh:2-6 is a 5-line file header. agentic.yaml:51-57 is a 7-line section header above the overrides that restates the golden AL values and the variant layout. The rule is labelled mandatory, so these break repo convention. They add no runtime defect.
Verification: nit. The trigger is that the PR merges as written. Nothing breaks at runtime. AGENTS.md:54-58 says "No header comments describing a file, topology, config, recipe, override, or section". nvidia-master.yaml (new lines 7428-7430) puts a 3-line header above mimov2.6flash-fp4-b200-vllm-agentic-dflash. mimov2.6flash-dflash-draft.sh lines 2-6 are a 5-line file header.
| ("dsv4dsparkprob", {"method": "dspark"}, "dsv4-pro-0813-dspark"), | ||
| ("dsv41flash", {"method": "dspark"}, "dsv41flash_dspark"), | ||
| ("musespark", {"method": "dflash"}, "musespark_dflash"), | ||
| ("mimov2.6flash", {"method": "dflash"}, "mimov2.6flash_dflash"), |
There was a problem hiding this comment.
🟡 nit (optional): The new test row adds no coverage and fails if someone renames the checked-in curve. It runs the same default f"{model}_{method}" branch as the musespark row. It also asserts that mimov2.6flash_dflash.yaml exists, and AGENTS.md says tests like that should be deleted. Fix: drop the row; test_every_committed_curve_is_valid already validates the new curve.
Why this was flagged
In curve_name (infx/golden_al_distribution/curves.py:50-67), the prefix "mimov2.6flash" with method "dflash" takes no special branch. It returns f"{model}_{method}", the same path the existing musespark, dsv41flash and minimaxm31flash rows already cover. test_curve_name_resolves_committed_curves:54 then asserts (GOLDEN_DIR / f"{expected}.yaml").is_file(), which only pins the checked-in file. The row fails if someone renames or retires the curve, even when no code changes. AGENTS.md Tests says tests that "pin checked-in config" or "duplicate a covered branch must be deleted" (questions 3 and 4). test_every_committed_curve_is_valid:18-28 already loads and range-checks every committed curve, including this one.
Verification: Trigger: the new parametrized row in test_golden_al_distribution.py:42 always runs. In curves.py curve_name, "dflash" is not "eagle" or "nextn", so the method stays "dflash". That is the same default path the existing ("musespark", {"method": "dflash"}) row at line 41 already covers. The row would fail if that curve were renamed or removed with no code change.
…at/mimov2.6flash-b200-sglang-agentic Resolve perf-changelog.yaml with prepare_perf_changelog_merge (main's bytes kept, the #4330 entry re-appended) and place the MiMo-V2.6-Flash MODELS.md row by date between MiniMax-M3.1-Flash and Muse Spark Open, pointing its speculative-decoding note at #4742. 中文:使用 prepare_perf_changelog_merge 解决 perf-changelog.yaml 冲突(保留 main 原有内容并重新追加 #4330 条目);将 MiMo-V2.6-Flash 在 MODELS.md 中的 行按日期放在 MiniMax-M3.1-Flash 与 Muse Spark Open 之间,并将其投机解码说明 指向 #4742。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BGggnGqhy9dML7NM9ow3Fm
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=41384884664 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=41384884664 |
…le gaps / [Klaud Cold] 让 MiMo DFlash vLLM 连接撑过 AIPerf 空闲间隔 Sweep 41377885108 lost TP4 c64 when two AgentX warmup requests hit "Connection reset by peer" on a live server: uvicorn's default 5 s keep-alive closed pooled connections just as AIPerf reused them. Set VLLM_HTTP_TIMEOUT_KEEP_ALIVE to 600, as the MiniMax-M3.1-Flash and Kimi vLLM AgentX recipes do. 中文:sweep 41377885108 中 TP4 c64 失败:两个 AgentX 预热请求在服务正常运行 时遇到 "Connection reset by peer",原因是 uvicorn 默认 5 秒的 keep-alive 恰在 AIPerf 复用连接池中的连接时将其关闭。现将 VLLM_HTTP_TIMEOUT_KEEP_ALIVE 设为 600,与 MiniMax-M3.1-Flash 和 Kimi 的 vLLM AgentX 配方一致。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BGggnGqhy9dML7NM9ow3Fm
Summary
Adds DFlash speculative decoding for XiaomiMiMo/MiMo-V2.6-Flash-RL AgentX on B200 with upstream vLLM, as a new key,
mimov2.6flash-fp4-b200-vllm-agentic-dflash.Stacked on #4331. This branch is built on #4331, which commits the golden curve
infx/golden_al_distribution/mimov2.6flash_dflash.yamlthis key needs, and it targetsmainso the sweep runs. Until #4331 merges, this diff also shows #4331's collector and curve. The recipe itself does not depend on #4330, the non-speculative SGLang and vLLM keys.What runs: 13 points, all with GPU-resident KV.
Seven draft tokens give the longest acceptance where decode is latency-bound. At higher concurrency, verifying 8 tokens per step costs more compute than it saves, so those points verify 4 (3 drafts plus the current token).
Server:
vllm/vllm-openai:v0.31.0(pinned by digest), as in the non-speculative vLLM key of #4330, following vLLM's MiMo-V2.6-Flash-RL recipe:speculative-configwithmethod: dflash, the checkpoint's own BF16dflash/drafter, and greedy drafting, which is how the golden curve was collected.cudagraph_mode: FULL_DECODE_ONLY, as in the recipe's DFlash example. Decode graphs cover concurrency × (1 + draft tokens), and admission stays at 2× concurrency for subagent fan-out.gpu-memory-utilization 0.95,max-model-len auto,max-num-batched-tokens 16384,mimoreasoning and tool-call parsers, thinking on.VLLM_HTTP_TIMEOUT_KEEP_ALIVE=600, so the server keeps idle connections open longer than AIPerf leaves its pooled connections idle. In the first sweep, 41377885108, TP4 c64 lost two warmup requests to connection resets under vLLM's default 5 s keep-alive while the server stayed healthy.Synthetic acceptance: the
infx/srt_slurmconnector reads the draft length from each point'sspeculative-configand setsrejection_sample_method: syntheticwith the golden AL forthinking_on. vLLM v0.31.0 supports synthetic rejection sampling fordflash.Draft path: vLLM loads a draft model only from a concrete local path, and
<repo>/dflashis not a Hub model, so the recipe names the drafter inside the shared hub cache:f657ba2f7d24c9adc5730675baaa2d34de878a49, the currentmain. Only README files changed after 2026-10-09, so the weights match those the [Klaud Cold] Add MiMo-V2.6-Flash B200 SGLang and vLLM AgentX recipes / [Klaud Cold] 新增 MiMo-V2.6-Flash B200 SGLang 与 vLLM AgentX 配方 #4330 sweep and the golden-curve collection used.mimov2.6flash-dflash-draft.sh, downloads that revision into/mnt/hf_hub_cachebefore vLLM starts, with retries as in the Kimi DSpark draft script. It fails the point ifdflash/config.jsonis still missing.Local validation:
validate_perf_changelogpasses.infx.matrix.changedlists only this key's 13 points.select_recipe, andbuild_overridesapplies AL 4.08 at 7 draft tokens and 2.97 at 3.infx/testssuite fails the same 31 tests on this branch and onmainhere, from missing local packages such assrtctl; CI runs them.Sweep: run 41384884664 passed all 13 points and the gsm8k eval (strict match 0.972 at TP4 c64). It needed two re-runs of the canary for infrastructure failures: Slurm job 24818 was cancelled from outside the workflow, and AIPerf then hit Hugging Face's API rate limit (429) while loading the trace dataset. No profiled request failed at any point; the dropped records are all warmup requests.
Against the non-speculative vLLM key from #4330 (run 39323832019), same image and flags:
Up to concurrency 8, interactivity rises about 3×, in line with the synthetic AL of 4.08. TP2 c32 is limited by KV capacity. Without speculation it already runs at 93% GPU KV usage with a 56% prefix hit rate; with the drafter also on the GPUs, usage reaches 99.8% and the hit rate falls to 33%, so DFlash gains little there.
Merge order: #4331 first. When #4330 is also merged, its MODELS.md row needs its "no speculative decoding" note replaced with this DFlash key, either here or in #4330, whichever lands second.
AI model disclosure
claude-opus-5-5), via Claude Code. No delegated agents for this PR.PR checklist and change type
Type of Change
Checklist
python -m infx.matrix.changed --base origin/mainlists affected config keys, I have appended one new entry covering them to the physical end ofinferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entriesinferencex-e2e/perf-changelog.yamlentry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs):full-sweep-fail-fast(recommended),full-sweep-enabled, ornon-canary-full-sweep-enabled. Optional modifiersall-evals,evals-only, andagentx-fastrequire a primary label; the last two block reuse while applied.OWNER/MEMBER/COLLABORATOR) has commented/use <run_id>(or the legacy/reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.中文
为 XiaomiMiMo/MiMo-V2.6-Flash-RL 在 B200 上基于上游 vLLM 的 AgentX 新增 DFlash 投机解码,对应新配置键
mimov2.6flash-fp4-b200-vllm-agentic-dflash。叠加在 #4331 之上。 本分支基于 #4331(提交了该配置键所需的黄金曲线
infx/golden_al_distribution/mimov2.6flash_dflash.yaml),目标分支为main,以便触发 sweep。#4331 合并前,本 PR 的 diff 也会显示 #4331 的采集脚本与曲线。配方本身不依赖 #4330(不带投机解码的 SGLang 与 vLLM 配置键)。运行内容: 13 个点,KV 缓存均保留在 GPU 上。
解码受延迟限制时,7 个草稿 token 的接受长度最长;并发较高时,每步验证 8 个 token 的计算开销超过收益,因此这些点每步验证 4 个(3 个草稿加当前 token)。
服务配置:
vllm/vllm-openai:v0.31.0(按 digest 固定),与 #4330 中不带投机解码的 vLLM 配置键相同,遵循 vLLM 的 MiMo-V2.6-Flash-RL 配方:speculative-config使用method: dflash、检查点自带的 BF16dflash/草稿模型与贪心草稿采样,与黄金曲线的采集方式一致。cudagraph_mode: FULL_DECODE_ONLY,与配方中的 DFlash 示例一致。解码 CUDA graph 覆盖“并发 ×(1 + 草稿 token 数)”,准入仍为并发的 2 倍以支持子 agent 扇出。gpu-memory-utilization 0.95、max-model-len auto、max-num-batched-tokens 16384、mimo推理与工具调用解析器,开启思考。VLLM_HTTP_TIMEOUT_KEEP_ALIVE=600,使服务端保持空闲连接的时间长于 AIPerf 连接池中连接的空闲时间。首次 sweep 41377885108 中,在 vLLM 默认 5 秒 keep-alive 下,TP4 c64 有两个预热请求因连接被重置而失败,而服务端始终正常。合成接受:
infx/srt_slurm连接器从每个点的speculative-config读取草稿长度,并按thinking_on的黄金 AL 设置rejection_sample_method: synthetic。vLLM v0.31.0 支持对dflash使用合成拒绝采样。草稿路径: vLLM 只能从具体的本地路径加载草稿模型,而
<repo>/dflash不是 Hub 模型,因此配方直接指向共享 hub 缓存中的草稿目录:f657ba2f7d24c9adc5730675baaa2d34de878a49(当前main)。2026-10-09 之后只有 README 变动,因此权重与 [Klaud Cold] Add MiMo-V2.6-Flash B200 SGLang and vLLM AgentX recipes / [Klaud Cold] 新增 MiMo-V2.6-Flash B200 SGLang 与 vLLM AgentX 配方 #4330 sweep 及黄金曲线采集所用的一致。mimov2.6flash-dflash-draft.sh,在 vLLM 启动前把该版本下载到/mnt/hf_hub_cache,并按 Kimi DSpark 草稿脚本的做法重试;若下载后dflash/config.json仍不存在,则该点失败。本地验证:
validate_perf_changelog通过。infx.matrix.changed只列出该配置键的 13 个点。select_recipe恰好绑定一个配方变体,build_overrides在 7 个草稿 token 时应用 AL 4.08,3 个时应用 2.97。infx/tests在本分支与main上本地都失败同样的 31 项,原因是缺少srtctl等本地依赖;CI 会运行这些测试。Sweep: 运行 41384884664 的 13 个点与 gsm8k 评估全部通过(TP4 c64 严格匹配 0.972)。canary 因基础设施问题重跑了两次:Slurm 作业 24818 在工作流之外被取消;随后 AIPerf 加载 trace 数据集时触发 Hugging Face API 限流(429)。所有点均无正式统计的请求失败,被丢弃的记录均为预热请求。
与 #4330 中不带投机解码的 vLLM 配置键(运行 39323832019,镜像与参数相同)对比:
并发不超过 8 时,交互性提升约 3 倍,与合成 AL 4.08 相符。TP2 c32 受 KV 容量限制:不带投机解码时 GPU KV 使用率已达 93%,前缀命中率为 56%;加入草稿模型后,使用率达 99.8%,命中率降至 33%,因此 DFlash 在该点提升有限。
合并顺序: 先合并 #4331。#4330 也合并后,其 MODELS.md 行中“不使用投机解码”的说明需替换为本 DFlash 配置键,由两者中后合并的一方修改。
AI 模型使用说明: Claude Opus 5.5(
claude-opus-5-5),通过 Claude Code 使用,本 PR 未委派其他 agent;负责 vLLM 源码与配方审阅、配方与配置编写、本地验证、sweep 监控与修复以及 PR 准备。改动类型: 新功能、配置变更。
🤖 Generated with Claude Code
https://claude.ai/code/session_01BGggnGqhy9dML7NM9ow3Fm