Skip to content

[GLM-5.3 GB300] add Dynamo-SGLang AgentX agg and disagg recipes with power / [GLM-5.3 GB300] 新增 Dynamo-SGLang AgentX 聚合与分离部署配方并采集功耗 - #4706

Open
edwingao28 wants to merge 11 commits into
mainfrom
fix/glm52-gb300-dynamo-power
Open

edwingao28 wants to merge 11 commits into
mainfrom
fix/glm52-gb300-dynamo-power

Conversation

@edwingao28

@edwingao28 edwingao28 commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

GLM-5.3 NVFP4 has AgentX coverage only on B300 SGLang (glm5.3-fp4-b300-sglang-agentic-mtp). No config runs nvidia/GLM-5.3-NVFP4 on GB300, and the gb300-nv runner has no path for its weights. GLM-5.3 therefore has no GB300 Dynamo-SGLang points and no measured GB300 power.

Changes:

  • glm5.3/sglang/gb300-fp4/agentx/ (new recipes), ported from the GLM-5.2 GB300 Dynamo-SGLang recipes:
    • agg-variants.yaml: one TP4 worker and one TP8 worker, each at conc 1.
    • disagg-variants.yaml: one to four PP4 prefill workers with TP4, TP8 or TP16 decode, six points at conc 30 to 264.
    • Both serve nvidia/GLM-5.3-NVFP4 at revision e3b8f9f2 with NEXTN MTP (3 steps, 4 draft tokens) and DRAM HiCache KV offload.
  • nvidia-master.yaml: adds glm5.3-fp4-gb300-dynamo-sglang-agentic-agg and glm5.3-fp4-gb300-dynamo-sglang-agentic-disagg on cluster:gb300-nv.
  • runners.yaml: maps GLM-5.3-NVFP4 to the data-models root on gb300-nv.
  • perf-changelog.yaml: one block listing both keys.

No launcher or infx/ code changes. All other configs are unchanged.

Results: sweep 41407333903, attempt 1, source 26d2411f3, GB300, AgentX agentic-coding traces. Every row comes from this run. The baseline is the published GLM-5.2 GB300 Dynamo-SGLang point with the same topology and concurrency, from 2026-10-20.

Mode GPUs (prefill / decode) Conc Output tok/s/GPU, GLM-5.2 → GLM-5.3 Δ Median interactivity (tok/s/user), GLM-5.2 → GLM-5.3 Δ GPU power, all GPUs (W) Energy (J/output token) gsm8k
agg 4 1 37.2 → 36.8 -1.3% 330 → 317 -3.8% 1,637 11.13 0.973
agg 8 1 19.4 → 19.1 -1.5% 380 → 358 -5.7% 2,836 18.57 0.975
disagg 4/20 30 65.0 → 64.4 -1.0% 318 → 302 -5.1% 10,836 8.42 0.974
disagg 4/12 40 145.4 → 145.0 -0.3% 228 → 229 +0.2% 8,680 4.99 0.976
disagg 4/8 56 303.8 → 292.9 -3.6% 139 → 121 -13.1% 7,218 3.09 0.976
disagg 12/8 216 982.5 → 984.1 +0.2% 101 → 101 -0.6% 13,040 1.66 0.977
disagg 16/16 256 627.6 → 611.7 -2.5% 125 → 114 -8.9% 18,651 1.91 0.976
disagg 12/8 264 1145.4 → 1129.0 -1.4% 97 → 96 -0.7% 14,362 1.60 0.976
  • GPU power is valid on 8 / 8 points.
  • Request errors: 0 on 7 of 8 points. The conc 256 point dropped 263 of 35,562 post-warmup requests (0.7%) as InvalidInferenceResultError.
  • The model differs from the baseline on every row, and the agg rows also use a newer image, so the deltas are not a controlled comparison.
  • The GLM-5.2 baseline has no power data, so energy has no comparison.

Testing:

  • python -m infx.matrix.changed --base origin/main at the current head e4904851a lists only the two new keys (8 points). Their point fingerprints equal those of run 41407333903, so the run still covers this head.

Limits:

  • The run used the provisional GLM-5.3 golden acceptance-length curve copied from GLM-5.2 (K=3 mean 2.99). Main now selects the measured glm-5.3-nvfp4 curve (mean 2.94, Select golden AL curves by verifier and add SPEED-Bench collectors / 按验证器选择黄金 AL 曲线并新增 SPEED-Bench 收集脚本 #4663). These points therefore assume a 1.7% higher mean acceptance length than a rerun on main would. GLM-5.3 points measured before Select golden AL curves by verifier and add SPEED-Bench collectors / 按验证器选择黄金 AL 曲线并新增 SPEED-Bench 收集脚本 #4663 carry the same offset.
  • The dashboard shows glm5.3 under the GLM-5.2 display model. These points join the published GLM-5.2 points in the GB300 Dynamo-SGLang series, as the two models' B300 SGLang points already do. The GLM-5.2 rows stay in the database.
    • Scatter charts: each run date gets its own frontier, so the GLM-5.2 frontier is unchanged in all 104 metric, axis and percentile views that have GLM-5.2 data. These points draw a second curve in the same color.
    • Throughput calculator: it fits one curve through both models' points. Across the shared interactivity range, its interpolated tok/s/chip moves by −2.8% to +3.8%, and its output tok/s/chip by −3.8% to +5.8%.
    • Best per SKU: on 20 power and energy metrics, this filter now picks GB300 Dynamo-SGLang instead of GB300 Dynamo-TRT. The GLM-5.2 GB300 Dynamo-SGLang points have no power data, so these are the first points in that series that compete on those metrics.
    • Checked by running production rows through the app's chart code, not on a rendered page.

Delivery: /use 41407333903 is posted. Merging reuses this run and starts no GPU job.

PR checklist and change type

Type of Change

  • Bug fix
  • New feature
  • Configuration change
  • Documentation update
  • Other (please describe)

Checklist

  • I have completed the AI model disclosure and kept it current
  • I have tested my changes locally
  • I have updated documentation if necessary
  • If python -m infx.matrix.changed --base origin/main lists affected config keys, I have appended one new entry covering them to the physical end of inferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entries
  • If this PR appends an inferencex-e2e/perf-changelog.yaml entry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs): full-sweep-fail-fast (recommended), full-sweep-enabled, or non-canary-full-sweep-enabled. Optional modifiers all-evals, evals-only, and agentx-fast require a primary label; the last two block reuse while applied.
  • Before merging via reuse, an authorized maintainer (OWNER/MEMBER/COLLABORATOR) has commented /use <run_id> (or the legacy /reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.

中文:采集 GLM-5.2 GB300 Dynamo-SGLang AgentX 功耗
@github-actions

github-actions Bot commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@edwingao28
edwingao28 force-pushed the fix/glm52-gb300-dynamo-power branch from 1238f20 to 1a66b16 Compare November 1, 2026 09:13

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Besides the inline finding, I checked the new GB300 GLM-5.2 PowerRule in infx/launch/drivers/srt/power.py. Its Match arguments are in the right order (prefixes, precisions, frameworks, then agentic=True). The rule also matches the existing GB200 GLM-5.2 rule exactly, which doesn't set require=True either, so leaving that flag off follows the existing pattern.

Extended reasoning...

I compared the new rule with Match's field order in infx/launch/policy.py and with the GB200 srt-multi lane, which already has an identical GLM-5.2 FP4 dynamo-sglang AgentX rule that doesn't set require.

Comment thread inferencex-e2e/perf-changelog.yaml Resolved
@edwingao28 edwingao28 added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Nov 1, 2026
@github-actions

github-actions Bot commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

中文:将 main 合并到 GLM-5.3 GB300 Dynamo-SGLang AgentX
中文:新增 GLM-5.3 FP4 GB300 Dynamo-SGLang AgentX 聚合配置
中文:新增 GLM-5.3 FP4 GB300 Dynamo-SGLang AgentX 分离式配置
中文:将 main 合并到 GLM-5.3 GB300 Dynamo-SGLang AgentX
@edwingao28 edwingao28 changed the title [PowerX] collect GLM-5.2 GB300 Dynamo-SGLang AgentX power / [PowerX] 采集 GLM-5.2 GB300 Dynamo-SGLang AgentX 功耗 [GLM-5.3 GB300] add Dynamo-SGLang AgentX agg and disagg recipes with power / [GLM-5.3 GB300] 新增 Dynamo-SGLang AgentX 聚合与分离部署配方并采集功耗 Nov 2, 2026
@edwingao28

Copy link
Copy Markdown
Collaborator Author

/use 41407333903

@github-actions

github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

@edwingao28 staged run 41407333903.

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

中文:更正 GLM-5.3 GB300 changelog 中的功耗规则描述。
中文:将 main 合并到 GLM-5.3 GB300 Dynamo-SGLang AgentX。
@edwingao28 edwingao28 removed the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Nov 3, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant