Repository navigation
[DNM, Experimental] Add GLM-5.3-Flash NVFP4 B300 AgentX / [DNM, Experimental] 添加 GLM-5.3-Flash NVFP4 B300 AgentX - #4711
functionstackx wants to merge 10 commits into
Conversation
|
Thanks for the contribution!
中文感谢你的贡献!
|
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=41300767755 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=41300767755 |
| glm5.3flash-fp4-b300-sglang-agentic-mtp: | ||
| image: lmsysorg/sglang:v0.5.22-cu130@sha256:e3a4f06001a29542fc4a8638536d9ffc5eb9c9942b08603c09b19a56eae4a1ae | ||
| model: nvidia/GLM-5.3-Flash-NVFP4 | ||
| model-prefix: glm5.3flash | ||
| runner: cluster:b300-dsxe | ||
| precision: fp4 | ||
| framework: sglang | ||
| multinode: false | ||
| srt-recipe-dir: glm5.3flash/sglang/b300-fp4-mtp | ||
| scenarios: | ||
| agentic-coding: | ||
| - search-space: | ||
| - { tp: 8, spec-decoding: mtp, kv-offloading: none, conc-list: [1, 2, 4, 8, 16, 32], srt-recipe: agentic.yaml } |
There was a problem hiding this comment.
🔴 Merging this adds a new benchmark that never gets results collected, because perf-changelog.yaml has no entry for it. AGENTS.md:87 says any change that alters what a benchmark runs needs a changelog entry for the affected config keys, since that entry triggers result collection. This PR adds the new config key glm5.3flash-fp4-b300-sglang-agentic-mtp and its recipe but leaves inferencex-e2e/perf-changelog.yaml untouched. Fix: append one English-only block at the tail of inferencex-e2e/perf-changelog.yaml. It should list config-keys glm5.3flash-fp4-b300-sglang-agentic-mtp and scenario-type agentic-coding, and keep every existing byte unchanged.
Why this was flagged
The diff adds the master entry glm5.3flash-fp4-b300-sglang-agentic-mtp at nvidia-master.yaml:8823-8835 and the recipe glm5.3flash/sglang/b300-fp4-mtp/agentic.yaml. Both define a benchmark that does not exist on the base branch. A grep of inferencex-e2e/perf-changelog.yaml finds no glm5.3flash key. The diff does not touch that file at all. AGENTS.md:87 says the changelog entry is what triggers result collection for affected config keys. Without it, this key's AgentX sweep (conc 1-32) is never scheduled, so no results appear after merge. The sibling glm5.3-fp4-b300-sglang-agentic-mtp has its own changelog block at perf-changelog.yaml:10869-10870, which shows the expected pattern.
Verification: The diff adds config key glm5.3flash-fp4-b300-sglang-agentic-mtp, but inferencex-e2e/perf-changelog.yaml is unchanged and has no glm5.3flash entry. AGENTS.md:87 says: "A change that alters what any benchmark runs needs a perf-changelog.yaml entry for the affected config keys; the entry triggers result collection." So the new conc 1-32 AgentX points will never be scheduled or collected.
There was a problem hiding this comment.
Fixed in commit 66bb1d6, after the reviewed initial commit 334faa1. The current head 6950741 includes exactly one append-only English changelog block for glm5.3flash-fp4-b300-sglang-agentic-mtp / agentic-coding, with this PR URL. The current-head changelog gate passed and full sweep is queued/running; GPU results are not complete yet.
中文:该问题针对初始提交 334faa1,已在后续提交 66bb1d6 修复。当前 head 6950741 包含一个仅追加的英文 changelog 条目,列出准确的 config key、agentic-coding 和本 PR 链接。当前 head 的 changelog 检查已通过,全量 sweep 已启动,等待 GPU 执行;尚未取得完整 GPU 结果。
|
The full sweep passed on current head This is qualified success, not error-free. Direct phase-separated JSONL counts show profiling errors 1/1/2/1/2/1 out of 305/456/700/1400/2663/4763 records, and warmup errors 11/15/34/51/105/231 out of 11/22/44/87/177/354 records. All are empty-content The canonical evaluation is GSM8K five-shot, 1,319/1,319 examples, no sample limit: strict accuracy 0.9711902956785443 and flexible accuracy 0.9704321455648218. The evaluation process fingerprint contains no synthetic-acceptance variables. The runtime artifact confirms the staged 中文:当前 head 本次结果并非零错误:按 JSONL 阶段拆分,profiling 错误数依次为 1/1/2/1/2/1,warmup 错误数为 11/15/34/51/105/231,均为空内容响应;profiling 错误涉及三条 trace,context-overflow skip 为零。GSM8K 使用五样本、完整 1319 题、无样本截断,strict/flexible 分别为 0.9711902956785443 / 0.9704321455648218,eval 不含 synthetic-acceptance 环境变量。运行日志及精确版本代码支持原始 draft 加载路径,但不是对运行中 tensor dtype 的直接观测。本次审计未重跑、restack 或合并 PR。 |
|
/use 41300767755 |
|
@Oseltamivir staging run 41300767755. Existing staged runs will be preserved; a completion link will be posted here. |
|
Generally most InferenceX submissions don't need to add changes to the infx package. On the off chance they do, generally the diff is small. Please prompt your agent to clean your PR to keep your diff to only necessary & minimal changes too. We have an increased code quality bar for changes to the infx package. 中文大多数 InferenceX 提交通常不需要修改 infx 包。即使确实需要,改动通常也很小。请让你的 agent 清理 PR,使改动只包含必要且最小的内容。infx 包的改动有更高的代码质量要求。 |
Add GLM-5.3-Flash NVFP4 AgentX coverage on B300 with TP8, EAGLE/MTP 5/1/6 and concurrency 1, 2, 4, 8, 16, 32. The recipe uses the official digest-pinned SGLang v0.5.22-cu130 image, TRT-LLM DSA prefill/decode, FP8 KV and FlashInfer TRT-LLM MoE from the pinned upstream NVIDIA NVFP4 cookbook.
This is the Flash hybrid MLA/DSA/KDA model, with the full 1,048,576-token context and unfiltered AgentX corpus. It does not change the separate GLM-5.3 configuration, patch the engine, download weights, or collect another acceptance curve. The shared connector selects the MTP5 thinking-on point from #4708; that common model curve was measured with FP8 vLLM in the collector introduced by #4701.
The model is
nvidia/GLM-5.3-Flash-NVFP4at revision1d00738646cbb0c6d0a9eb403dac5f366f1473a7, as reported by the checkpoint metadata. Its quantization exclusions cover embedded draft layer 45; the recipe leaves draft loading to the pinned upstream implementation and explicitly setsSGLANG_NVFP4_CKPT_FP8_NEXTN_MOE=0andSGLANG_GLM_NEXTN_MOE_PTPC=0.Draft precision qualification: the runtime fingerprint identifies build
baa9befa381511cd4d56f83980951e5a45fffc6d, and the exact-build inherited NextN constructor clears themodelopt_fp4decoder quantization configuration before decoder construction. Both conversion flags remain0, and no draft quantization override is introduced. This exact-build source-path evidence and checkpoint metadata support draft-as-shipped loading, but do not directly observe live tensor dtypes; the global NextN wrapper's NVFP4 log is not a decoder-precision measurement.The successful runtime artifact confirms the staged
/data/models/GLM-5.3-Flash-NVFP4:/modelmount, full 1,048,576-token context, TP8/EP1, EAGLE 5/1/6, the declared model revision, and native AL3.603536. No weight download was added.Stack: this ready PR targets
feat/glm53flash-golden-al(#4708), which depends on collector #4701. It is not intended to merge before those prerequisites.Validation: all 102 targeted workload/synthetic-acceptance tests passed, and the full sweep for current head 6950741 completed successfully across c1/c2/c4/c8/c16/c32, independent c32 evaluation, and result collection. The regenerated current-head matrix matches all six collected recipe fingerprints (
f3e422e1b1c5fe483c52f7dad4985b24ea80c74ba72cf34ee4af1c8881a9f4c4). Every point reportspower_valid=1, the full 393-entry trace corpus, a 3,600-second profiling command without trace filtering, and zero context-overflow skips.The result is not error-free. Direct phase-separated JSONL accounting finds 8 profiling errors out of 10,287 records (10,279 successful) and 447 warmup errors out of 695 records. Profiling errors by concurrency are 1/1/2/1/2/1; warmup errors are 11/15/34/51/105/231. All are empty-content
InvalidInferenceResultError; the profiling errors affect three traces and their cause remains unresolved. Combinedrecords_error_droppedis not a profiling-error count. The qualified audit comment records the per-point details.The canonical GSM8K evaluation uses five shots, all 1,319 examples, and no sample limit. Strict accuracy is 1281/1319 (
0.9711902956785443); flexible accuracy is 1280/1319 (0.9704321455648218). The evaluation runtime fingerprint contains no synthetic-acceptance variables and retains both prohibited conversion flags at0.中文
为 B300 添加 GLM-5.3-Flash NVFP4 AgentX 配置,使用 TP8、EAGLE/MTP 5/1/6,以及并发 1、2、4、8、16、32。官方 SGLang v0.5.22-cu130 镜像按 digest 固定,采用上游 cookbook 的 TRT-LLM DSA、FP8 KV 和 FlashInfer TRT-LLM MoE。
保留完整 1,048,576-token 上下文和未截断的 AgentX 数据集,不修改独立的 GLM-5.3 配置,不打引擎补丁,不下载权重,也不重新采集 AL。共享连接器读取 #4708 的模型公共 MTP 曲线;该曲线由 #4701 引入的 FP8 vLLM collector 实测。
模型为
nvidia/GLM-5.3-Flash-NVFP4,revision 为1d00738646cbb0c6d0a9eb403dac5f366f1473a7。运行确认SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE和SGLANG_GLM_NEXTN_MOE_PTPC均为0,没有显式 draft 量化覆盖。精确 buildbaa9befa381511cd4d56f83980951e5a45fffc6d的继承 NextN 构造函数在建立 decoder 前清除modelopt_fp4量化配置;该代码路径和 checkpoint 元数据支持原始 draft 加载,但不是对运行中 tensor dtype 的直接观测,也不能把全局 wrapper 的 NVFP4 日志当作 decoder 精度证明。成功运行已确认
/data/models/GLM-5.3-Flash-NVFP4:/model挂载、完整 1,048,576-token 上下文、TP8/EP1、EAGLE 5/1/6 和 AL3.603536。当前 head6950741的六个并发点、独立 canonical eval 及结果收集均通过;当前矩阵和全部 artifact fingerprint 一致,所有点power_valid=1,使用完整 393-entry 数据集、3600 秒 profiling,未过滤 trace,context-overflow skip 为零。结果并非零错误:profiling 共 10,287 条记录中 8 条错误、10,279 条成功,warmup 695 条记录中 447 条错误。按并发顺序,profiling 错误为 1/1/2/1/2/1,warmup 错误为 11/15/34/51/105/231;均为空内容响应,profiling 涉及三条 trace,根因仍未解决。不能把合并阶段的
records_error_dropped当作 profiling 错误数。GSM8K 使用五样本、完整 1,319 题、无样本截断,strict 为 1281/1319,flexible 为 1280/1319;eval 进程不含 synthetic-acceptance 环境变量。PR 仍以 #4708 分支为目标并依赖 #4701,保持原 head 和唯一 full-sweep-fail-fast 标签;本次审计未重跑、restack 或合并,前置 PR 合并前不应合并。
AI model disclosure
GPT-6 Astra coordinated the four sibling PRs. GPT-6 Astra delegated agents implemented this B300 NVFP4 recipe and cross-checked checkpoint, image, and draft-precision evidence with the B200 NVFP4 agent.