[Klaud Cold] Update GLM-5.2 FP8 MI355X SGLang AgentX MTP image to v0.5.22 / [Klaud Cold] 将 GLM-5.2 FP8 MI355X SGLang AgentX MTP 镜像更新至 v0.5.22 by Klaud-Cold · Pull Request #4518 · SemiAnalysisAI/InferenceX

Klaud-Cold · 2026-10-25T20:22:23Z

Goal: Update SGLang image from lmsysorg/sglang-rocm:v0.5.19-rocm720-mi35x-20260917 to lmsysorg/sglang:v0.5.22-rocm720-mi35x@sha256:c4b61360b251d62bb17da63f175d571ab22479eb46a144548d79f3ed3998ea11.
Baseline: 2026-09-19 · lmsysorg/sglang-rocm:v0.5.19-rocm720-mi35x-20260917
AgentX · TP8/EP1 · Mean latency · Sources: API 1, API 2, API 3, API 4

Concurrency	Total tok/s/GPU ↑	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
1	2,799.42	8.3	1,194.29	5.95
2	2,784.99	10.78	874.37	6.43
4	4,012.14	14.57	841.78	6.82
8	7,042.11	26.78	746.79	8.73
10	8,172.44	33.19	988.41	10.19
12	7,710.19	32.25	1,221.74	12.08
16	9,244.63	36.48	2,326.62	23.02

Eval	Score ↑	Samples
gsm8k/em_strict · c16	96.66%	1,319

Stop: Close this PR at any time to stop this candidate now; Klaud cancels its runs and deletes its branch.

中文

**目标：**将 SGLang 镜像从 lmsysorg/sglang-rocm:v0.5.19-rocm720-mi35x-20260917 更新至 lmsysorg/sglang:v0.5.22-rocm720-mi35x@sha256:c4b61360b251d62bb17da63f175d571ab22479eb46a144548d79f3ed3998ea11。
**基线：**2026-09-19 · lmsysorg/sglang-rocm:v0.5.19-rocm720-mi35x-20260917
AgentX · TP8/EP1 · 平均延迟 · 来源： API 1, API 2, API 3, API 4；数值及异常说明见上表。
**停止：**随时关闭此 PR 即可立即停止该候选：Klaud 会取消其运行并删除其分支。

…0.5.22 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

github-actions · 2026-10-25T20:22:31Z

Thanks for the contribution!

Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.

中文

感谢你的贡献！

**审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
**PR 验证：**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目，并且恰好带有一个主标签时，才会运行扫描：full-sweep-fail-fast（强烈推荐；canary 加逐矩阵 fail-fast）、full-sweep-enabled（有 canary；矩阵任务在失败后继续运行）或 non-canary-full-sweep-enabled（无 canary，也无 fail-fast）。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
**合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

Klaud-Cold · 2026-10-25T20:24:36Z

Initial attempt · Passed · Run 40230541980 / attempt 1 · 2026-10-25 22:31 UTC
lmsysorg/sglang:v0.5.22-rocm720-mi35x@sha256:c4b61360b251d62bb17da63f175d571ab22479eb46a144548d79f3ed3998ea11 · 88588aaf353a · AgentX · TP8/EP1 · Mean latency
Change: Set the master image to the official SGLang v0.5.22 ROCm 7.2 MI35X release, pinned by digest; source diff from main a9fb1c3238 (old image) to tag v0.5.22 keeps every recipe flag and env (tilelang DSA backends, glm47/glm45 parsers, EAGLE MTP, HiCache write_through_selective/page_first_direct, SGLANG_OPT_USE_TOPK_V2) with unchanged semantics, while AITER moves 4ad99832 to e7d2453f and v0.5.22 newly enables AITER allreduce fusion on decode for GlmMoeDsaForCausalLM on ROCm by default.

Concurrency	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
1	14.61 (N/A)	936.21 (N/A)	5.12 (N/A)
8	42.79 (N/A)	626.02 (N/A)	8.11 (N/A)

Note: All rows: Δ N/A: no matched baseline.

Eval	Score ↑	Samples
gsm8k/em_strict · c16	96.51% (-0.15 pp)	1,319 each

Next: Smoke on ref 60df019 passed (c1, c8 HiCache, zero request errors, gsm8k 96.51% at c16); append the perf-changelog entry and dispatch the full-sweep-fail-fast final sweep.

中文

初次尝试 · 已通过 · Run 40230541980 / attempt 1 · 2026-10-25 22:31 UTC
lmsysorg/sglang:v0.5.22-rocm720-mi35x@sha256:c4b61360b251d62bb17da63f175d571ab22479eb46a144548d79f3ed3998ea11 · 88588aaf353a · AgentX · TP8/EP1 · 平均延迟
**变更：**将主镜像设为官方 SGLang v0.5.22 ROCm 7.2 MI35X 发布版并按 digest 固定；从旧镜像的 main a9fb1c3238 到 tag v0.5.22 的源码对比显示，所有 recipe 参数和环境变量（tilelang DSA 后端、glm47/glm45 解析器、EAGLE MTP、HiCache write_through_selective/page_first_direct、SGLANG_OPT_USE_TOPK_V2）均保留且语义不变，AITER 从 4ad99832 升级到 e7d2453f，且 v0.5.22 在 ROCm 上默认为 GlmMoeDsaForCausalLM 的 decode 启用 AITER allreduce fusion。实测数值及异常说明见上表。
**下一步：**ref 60df019 的 smoke 运行已通过（c1、c8 HiCache，零请求错误，c16 gsm8k 96.51%）；接下来追加 perf-changelog 条目并启动 full-sweep-fail-fast 最终扫描。

Klaud-Cold · 2026-10-25T20:50:31Z

parked · Session 1 is waiting on 40230541980. Recovery reruns preempted runs and starts the next session once they end.

中文

parked · 第 1 个会话正在等待 40230541980。恢复流程会重跑被抢占的运行，并在运行结束后启动下一个会话。

Klaud-Cold · 2026-10-25T22:29:42Z

resumed · The session ended before finishing; session 2 continues it in a new run of its planner.

中文

resumed · 会话未完成即已结束；第 2 个会话将在其规划运行的新运行中继续。

… v0.5.22 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Klaud-Cold · 2026-10-25T22:34:39Z

Final full sweep · Queued · Run 40240738227 / attempt 1 · 2026-10-25 22:34 UTC
lmsysorg/sglang:v0.5.22-rocm720-mi35x@sha256:c4b61360b251d62bb17da63f175d571ab22479eb46a144548d79f3ed3998ea11 · f5c1fdf25ca4 · AgentX · TP8/EP1 · Mean latency
Change: Append the perf-changelog entry for the SGLang v0.5.22 image and run the full-sweep-fail-fast final sweep over every AgentX point.

Concurrency	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
1	N/A	N/A	N/A
2	N/A	N/A	N/A
4	N/A	N/A	N/A
8	N/A	N/A	N/A
10	N/A	N/A	N/A
12	N/A	N/A	N/A
16	N/A	N/A	N/A

Note: All rows: pending.
Note: All rows: request errors unavailable.
Note: All rows: Δ N/A: point unavailable.

Next: Wait for the queued final sweep (c1-c16 and the c16 gsm8k eval) to complete, then verify artifacts against the frozen baseline.

中文

最终完整 sweep · 排队中 · Run 40240738227 / attempt 1 · 2026-10-25 22:34 UTC
lmsysorg/sglang:v0.5.22-rocm720-mi35x@sha256:c4b61360b251d62bb17da63f175d571ab22479eb46a144548d79f3ed3998ea11 · f5c1fdf25ca4 · AgentX · TP8/EP1 · 平均延迟
**变更：**为 SGLang v0.5.22 镜像追加 perf-changelog 条目，并对全部 AgentX 点运行 full-sweep-fail-fast 最终扫描。实测数值及异常说明见上表。
**下一步：**等待已排队的最终扫描（c1-c16 及 c16 gsm8k 评测）完成，然后对照冻结的基线核验产物。

Klaud-Cold · 2026-10-25T22:34:47Z

parked · Session 2 is waiting on 40240738227. Recovery reruns preempted runs and starts the next session once they end.

中文

parked · 第 2 个会话正在等待 40240738227。恢复流程会重跑被抢占的运行，并在运行结束后启动下一个会话。

github-actions · 2026-10-26T02:21:11Z

View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=40462652710

View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=40462652710

Klaud-Cold · 2026-10-27T06:39:21Z

resumed · The session ended before finishing; session 3 continues it in a new run of its planner.

中文

resumed · 会话未完成即已结束；第 3 个会话将在其规划运行的新运行中继续。

…355952-8c92c0a196b4eef1 # Conflicts: # inferencex-e2e/perf-changelog.yaml

Klaud-Cold · 2026-10-27T06:43:25Z

Final full sweep · Failed · Run 40240738227 / attempt 4 · 2026-10-27 06:43 UTC
lmsysorg/sglang:v0.5.22-rocm720-mi35x@sha256:c4b61360b251d62bb17da63f175d571ab22479eb46a144548d79f3ed3998ea11 · f5c1fdf25ca4 · AgentX · TP8/EP1 · Mean latency
Change: Final sweep on f5c1fdf after three preempted attempts; c10 failed in the mi355x-amds host_setup hook because node mia1-p01-g15 still held 23% VRAM from a prior job after the 15 min drain wait, before the SGLang server started, and fail-fast cancelled the other points.

Concurrency	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
1	N/A	N/A	N/A
2	N/A	N/A	N/A
4	N/A	N/A	N/A
8	N/A	N/A	N/A
10	N/A	N/A	N/A
12	N/A	N/A	N/A
16	N/A	N/A	N/A

Note: c1, c2, c4, c8, c12, c16: cancelled.
Note: All rows: request errors unavailable.
Note: All rows: Δ N/A: point unavailable.
Note: c10: failed.

Eval	Score ↑	Samples
gsm8k/em_strict · c16	96.97% (+0.30 pp)	1,319 each

Next: This is a node-state infrastructure failure, not an image failure (gsm8k c16 still passed at 96.97%); main also appended perf-changelog entries, so merge main and rerun the final sweep on the new head without using a repair.

中文

最终完整 sweep · 失败 · Run 40240738227 / attempt 4 · 2026-10-27 06:43 UTC
lmsysorg/sglang:v0.5.22-rocm720-mi35x@sha256:c4b61360b251d62bb17da63f175d571ab22479eb46a144548d79f3ed3998ea11 · f5c1fdf25ca4 · AgentX · TP8/EP1 · 平均延迟
**变更：**f5c1fdf2 上的最终扫描在三次被抢占后运行；c10 在 SGLang 服务启动前于 mi355x-amds host_setup 钩子中失败，原因是节点 mia1-p01-g15 在 15 分钟排空等待后仍保留前一作业 23% 的显存，fail-fast 随后取消了其他点。实测数值及异常说明见上表。
**下一步：**这是节点状态导致的基础设施失败，并非镜像问题（c16 gsm8k 仍以 96.97% 通过）；main 也追加了 perf-changelog 条目，因此合并 main 并在新 head 上重新运行最终扫描，不消耗修复次数。

Klaud-Cold · 2026-10-27T06:43:35Z

Final full sweep · Queued · Run 40462652710 / attempt 1 · 2026-10-27 06:43 UTC
lmsysorg/sglang:v0.5.22-rocm720-mi35x@sha256:c4b61360b251d62bb17da63f175d571ab22479eb46a144548d79f3ed3998ea11 · dabf564932ac · AgentX · TP8/EP1 · Mean latency
Change: Merge main (new perf-changelog entries) keeping this entry at the physical tail, and rerun the full-sweep-fail-fast final sweep over every AgentX point.

Concurrency	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
1	N/A	N/A	N/A
2	N/A	N/A	N/A
4	N/A	N/A	N/A
8	N/A	N/A	N/A
10	N/A	N/A	N/A
12	N/A	N/A	N/A
16	N/A	N/A	N/A

Note: All rows: pending.
Note: All rows: request errors unavailable.
Note: All rows: Δ N/A: point unavailable.

Next: Wait for the queued final sweep (c1-c16 and the c16 gsm8k eval) to complete, then verify artifacts against the frozen baseline.

中文

最终完整 sweep · 排队中 · Run 40462652710 / attempt 1 · 2026-10-27 06:43 UTC
lmsysorg/sglang:v0.5.22-rocm720-mi35x@sha256:c4b61360b251d62bb17da63f175d571ab22479eb46a144548d79f3ed3998ea11 · dabf564932ac · AgentX · TP8/EP1 · 平均延迟
**变更：**合并 main（新增 perf-changelog 条目）并保持本条目位于文件末尾，然后对全部 AgentX 点重新运行 full-sweep-fail-fast 最终扫描。实测数值及异常说明见上表。
**下一步：**等待已排队的最终扫描（c1-c16 及 c16 gsm8k 评测）完成，然后对照冻结的基线核验产物。

Klaud-Cold · 2026-10-27T06:43:53Z

parked · Session 3 is waiting on 40462652710. Recovery reruns preempted runs and starts the next session once they end.

中文

parked · 第 3 个会话正在等待 40462652710。恢复流程会重跑被抢占的运行，并在运行结束后启动下一个会话。

Klaud-Cold · 2026-10-27T14:45:01Z

resumed · The session ended before finishing; session 4 continues it in a new run of its planner.

中文

resumed · 会话未完成即已结束；第 4 个会话将在其规划运行的新运行中继续。

Klaud-Cold · 2026-10-27T14:48:06Z

Final full sweep · Passed · Run 40462652710 / attempt 3 · 2026-10-27 17:50 UTC
lmsysorg/sglang:v0.5.22-rocm720-mi35x@sha256:c4b61360b251d62bb17da63f175d571ab22479eb46a144548d79f3ed3998ea11 · dabf564932ac · AgentX · TP8/EP1 · Mean latency
Change: Validate the complete updated-image family at this head.

Concurrency	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
1	14.68 (+76.9%)	860.52 (-27.9%)	5.07 (-14.8%)
2	18.25 (+69.4%)	523.32 (-40.1%)	5.56 (-13.5%)
4	23.44 (+60.8%)	522.92 (-37.9%)	5.99 (-12.2%)
8	42.78 (+59.7%)	498.02 (-33.3%)	8.27 (-5.3%)
10	52.38 (+57.8%)	620.1 (-37.3%)	9.67 (-5.1%)
12	50.48 (+56.5%)	650.51 (-46.8%)	10.19 (-15.6%)
16	64.24 (+76.1%)	1,165.09 (-49.9%)	16.2 (-29.6%)

Note: c12: 2 request errors.
Note: c16: 1 request error.

Eval	Score ↑	Samples
gsm8k/em_strict · c16	97.19% (+0.53 pp)	1,319 each

Next: Mark ready for maintainer review.

中文

最终完整 sweep · 已通过 · Run 40462652710 / attempt 3 · 2026-10-27 17:50 UTC
lmsysorg/sglang:v0.5.22-rocm720-mi35x@sha256:c4b61360b251d62bb17da63f175d571ab22479eb46a144548d79f3ed3998ea11 · dabf564932ac · AgentX · TP8/EP1 · 平均延迟
**变更：**验证此提交更新镜像后的完整配置族。实测数值及异常说明见上表。
**下一步：**标记为就绪，等待维护者审查。

Klaud-Cold · 2026-10-27T14:48:21Z

parked · Session 4 is waiting on 40462652710. Recovery reruns preempted runs and starts the next session once they end.

中文

parked · 第 4 个会话正在等待 40462652710。恢复流程会重跑被抢占的运行，并在运行结束后启动下一个会话。

…eep]

adibarra · 2026-11-13T19:27:33Z

not minimal changes only

chore(glm5.2): update GLM-5.2 FP8 MI355X SGLang AgentX MTP image to v…

60df019

…0.5.22 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

github-project-automation Bot added this to InferenceX Board Oct 25, 2026

chore(glm5.2): add perf-changelog entry for GLM-5.2 FP8 MI355X SGLang…

f5c1fdf

… v0.5.22 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Klaud-Cold added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 25, 2026

github-actions Bot added klaud-in-progress Klaud is working on this candidate or will resume it and removed klaud-parked Klaud parked until its queued or running benchmark runs end labels Oct 27, 2026

Merge remote-tracking branch 'origin/main' into klaud/auto-5c762659ac…

dabf564

…355952-8c92c0a196b4eef1 # Conflicts: # inferencex-e2e/perf-changelog.yaml

github-actions Bot added klaud-parked Klaud parked until its queued or running benchmark runs end and removed klaud-in-progress Klaud is working on this candidate or will resume it labels Oct 27, 2026

github-actions Bot added klaud-in-progress Klaud is working on this candidate or will resume it and removed klaud-parked Klaud parked until its queued or running benchmark runs end labels Oct 27, 2026

github-actions Bot added 27 commits November 7, 2026 00:38

Merge main into klaud/auto-5c762659ac355952-8c92c0a196b4eef1 [skip-sw…

0763c0b

…eep]

Merge main into klaud/auto-5c762659ac355952-8c92c0a196b4eef1 [skip-sw…

9ae170a

…eep]

Merge main into klaud/auto-5c762659ac355952-8c92c0a196b4eef1 [skip-sw…

dd22f60

…eep]

Merge main into klaud/auto-5c762659ac355952-8c92c0a196b4eef1 [skip-sw…

fff6faf

…eep]

Merge main into klaud/auto-5c762659ac355952-8c92c0a196b4eef1 [skip-sw…

2e09e6f

…eep]

Merge main into klaud/auto-5c762659ac355952-8c92c0a196b4eef1 [skip-sw…

bc537b0

…eep]

Merge main into klaud/auto-5c762659ac355952-8c92c0a196b4eef1 [skip-sw…

3f7ccd6

…eep]

Merge main into klaud/auto-5c762659ac355952-8c92c0a196b4eef1 [skip-sw…

c923089

…eep]

Merge main into klaud/auto-5c762659ac355952-8c92c0a196b4eef1 [skip-sw…

30aa030

…eep]

Merge main into klaud/auto-5c762659ac355952-8c92c0a196b4eef1 [skip-sw…

451bfa3

…eep]

Merge main into klaud/auto-5c762659ac355952-8c92c0a196b4eef1 [skip-sw…

9193b30

…eep]

Merge main into klaud/auto-5c762659ac355952-8c92c0a196b4eef1 [skip-sw…

4fdbd3f

…eep]

Merge main into klaud/auto-5c762659ac355952-8c92c0a196b4eef1 [skip-sw…

97e092e

…eep]

Merge main into klaud/auto-5c762659ac355952-8c92c0a196b4eef1 [skip-sw…

fa2b94e

…eep]

Merge main into klaud/auto-5c762659ac355952-8c92c0a196b4eef1 [skip-sw…

fe50b75

…eep]

Merge main into klaud/auto-5c762659ac355952-8c92c0a196b4eef1 [skip-sw…

c5f953f

…eep]

Merge main into klaud/auto-5c762659ac355952-8c92c0a196b4eef1 [skip-sw…

9a6404c

…eep]

Merge main into klaud/auto-5c762659ac355952-8c92c0a196b4eef1 [skip-sw…

54b4fd1

…eep]

Merge main into klaud/auto-5c762659ac355952-8c92c0a196b4eef1 [skip-sw…

9211fc1

…eep]

Merge main into klaud/auto-5c762659ac355952-8c92c0a196b4eef1 [skip-sw…

903552e

…eep]

Merge main into klaud/auto-5c762659ac355952-8c92c0a196b4eef1 [skip-sw…

75f3400

…eep]

Merge main into klaud/auto-5c762659ac355952-8c92c0a196b4eef1 [skip-sw…

f95639d

…eep]

Merge main into klaud/auto-5c762659ac355952-8c92c0a196b4eef1 [skip-sw…

f8a1ecd

…eep]

Merge main into klaud/auto-5c762659ac355952-8c92c0a196b4eef1 [skip-sw…

10a69fd

…eep]

Merge main into klaud/auto-5c762659ac355952-8c92c0a196b4eef1 [skip-sw…

723bd78

…eep]

Merge main into klaud/auto-5c762659ac355952-8c92c0a196b4eef1 [skip-sw…

529085d

…eep]

Merge main into klaud/auto-5c762659ac355952-8c92c0a196b4eef1 [skip-sw…

0ff822b

…eep]

adibarra closed this Nov 13, 2026

github-project-automation Bot moved this to Done in InferenceX Board Nov 13, 2026

Conversation

Klaud-Cold commented Oct 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 25, 2026

Uh oh!

Klaud-Cold commented Oct 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Klaud-Cold commented Oct 25, 2026

Uh oh!

Klaud-Cold commented Oct 25, 2026

Uh oh!

Klaud-Cold commented Oct 25, 2026

Uh oh!

Klaud-Cold commented Oct 25, 2026

Uh oh!

github-actions Bot commented Oct 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Klaud-Cold commented Oct 27, 2026

Uh oh!

Klaud-Cold commented Oct 27, 2026

Uh oh!

Klaud-Cold commented Oct 27, 2026

Uh oh!

Klaud-Cold commented Oct 27, 2026

Uh oh!

Klaud-Cold commented Oct 27, 2026

Uh oh!

Klaud-Cold commented Oct 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Klaud-Cold commented Oct 27, 2026

Uh oh!

adibarra commented Nov 13, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Klaud-Cold commented Oct 25, 2026 •

edited

Loading

Klaud-Cold commented Oct 25, 2026 •

edited

Loading

github-actions Bot commented Oct 26, 2026 •

edited

Loading

Klaud-Cold commented Oct 27, 2026 •

edited

Loading