Skip to content

perf(amd): DSv4.1-Flash ATOM AgentX TP4 c1 only on the 1111 nightly / DSv4.1-Flash ATOM AgentX 1111 nightly 只测 TP4 c1 - #5012

Draft
yhl-amd wants to merge 39 commits into
mainfrom
perf/dsv41flash-atom-1111-tp4c8
Draft

yhl-amd wants to merge 39 commits into
mainfrom
perf/dsv41flash-atom-1111-tp4c8

Conversation

@yhl-amd

@yhl-amd yhl-amd commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

Draft. Single-point check of DeepSeek-V4.1-Flash on ATOM AgentX (MI355X) with the 1111 nightly: plain TP4 at concurrency 1.

只测一个点:DSv4.1-Flash ATOM AgentX(MI355X),1111 nightly,plain TP4 c1。

🤖 Generated with Claude Code

yhl-amd and others added 30 commits October 14, 2026 00:56
…ightly

Move dsv41flash-fp4-mi355x-atom-agentic-dspark to
rocm/atom-dev:nightly_202610131505 (ATOM 661ef7661, with ROCm/ATOM#2376
V4.1 DPA and prefill TBO). Run TP2 c32/64/128/256 and TP4 c32 with DP
attention, prefill TBO, DP session affinity and the prefill delayer; drop
TP2 c6.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/dsv41flash/atom/mi355x-fp4-mtp/agentic.yaml
#	inferencex-e2e/configs/amd-master.yaml
#	inferencex-e2e/perf-changelog.yaml
…1013-dpa

# Conflicts:
#	inferencex-e2e/perf-changelog.yaml
rocm/atom-dev:nightly_202610221638 (sha256:32c81e56e468...; ATOM
0.1.9.dev11+gce46f6877, ROCm 7.2.4).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… without TBO

Merge origin/main, move the image to rocm/atom-dev:nightly_202610231458
(sha256:05322b86...) and drop --enable-tbo from the DP-attention points:
TBO does not support SWA bounded replay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Use the ATOM 2c772a5 release image (ROCm/ATOM run 39986380362,
sha256:01a19e8b...) instead of nightly_202610231458.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
max-num-seqs 128 queued half of the 256 sessions (TTFT 490 s); raise it to
256 and capture a 256 decode graph on that point.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…seqs as B300 vLLM

Move to the ATOM 11a2d56 release image (ROCm/ATOM run 40072622970,
sha256:42716923...) and set max-num-seqs per point (per DP rank) like the
B300 vLLM AgentX recipe: 1 at c1, 2x concurrency for c2-c16, concurrency
from c32 up; capture sizes stop at each point's max-num-seqs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1013-dpa

# Conflicts:
#	inferencex-e2e/perf-changelog.yaml
max-num-seqs is twice the concurrency at c1 too (TP2 and TP4).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop every max-num-seqs setting; decode graphs go back to the base list
(sizes through 32 plus 48, 64, 128), with 256 added on the c256 points.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…OM AgentX on nightly_202610261558

Image rocm/atom-dev:nightly_202610261558 (ATOM 713eb06, digest
sha256:57477e7a...) adds ROCm/ATOM#2696/#2697, which fix the hang every
DP-attention point hit on nightly_202610240802, and #2719 (trace labels).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t c64, DEP2 at c128/c256

Plain TP2 / TP4 at concurrency 1-32: c1 keeps the ROCm/ATOM#2659 recipe,
c2-c32 turn on the prefill delayer (decode interval 192 at c2-c8, 96 at
c16, 30 / 32 at c32). DTP2 (DP attention without TBO) takes c64 and DEP2
c128 and c256; plain TP at 64 and above is dropped. The server keeps idle
connections for 900 s, past uvicorn's 5 s default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1013-dpa

# Conflicts:
#	inferencex-e2e/perf-changelog.yaml
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	inferencex-e2e/perf-changelog.yaml
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	inferencex-e2e/perf-changelog.yaml
…val 40

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1013-dpa

# Conflicts:
#	inferencex-e2e/perf-changelog.yaml
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1013-dpa

# Conflicts:
#	inferencex-e2e/perf-changelog.yaml
seungrokj and others added 3 commits November 11, 2026 16:10
…e diff

- Update the image from rocm/atom-dev:nightly_202610121516 to nightly_202611101510 (ATOM e5b74bc86, ROCm 7.2.4) in the master config, and the AToMesh frontend container_image from sha256:a60811a8... to sha256:160a4ec4..., the same image.
- Replace plain TP2 at concurrency [1, 2, 4, 6, 8, 16, 32, 64] and plain TP4 at [1, 8, 32] with plain TP2 at [1, 2, 16, 32], DTP2 at 64, DEP2 at [128, 256] and plain TP4 at [1, 2, 4, 8]. TP2 c4, c6, c8 and c64 and TP4 c32 are dropped; TP4 c2 and c4 are new.
- Switch the KV cache from BF16 to MXFP8 at every point; the index cache stays FP4.
- Drop max-num-seqs 128. Concurrency 1 runs max-num-seqs 2, decode graphs [1, 2] and ATOM_PREFILL_DECODE_INTERVAL=1000000, the C1 recipe of ATOM's recipes/Kimi-K3.md (ROCm/ATOM#2659). The c256 point also captures 256.
- Set ATOM_PREFILL_DECODE_INTERVAL per point: 192 at TP2 c2 and TP4 c2-c8, 96 at TP2 c16, 40 at TP2 c32 and 10 at the DP points.
- The DP points add dp-load-balance least_tokens, ATOM_DP_SESSION_AFFINITY=1, ATOM_DP_LB_REQ_EQUIV=512 and GPU_MAX_HW_QUEUES=5, with AIPerf session ids as X-Dynamo-Session-Id.
- Every point adds timeout-keep-alive 900.
- The new ATOM brings V4.1 DP attention (#2376), the V4.1 MXFP8 KV cache with DSpark (#2725), DP prefill balancing and graph replay (#2551, #2696, #2697) and the default HIP Mega-MHC kernel (#2568).
- No change to the DSpark draft's weight precision; its KV cache follows the target from BF16 to MXFP8.

Also replace the recipe and master-config header comments with line comments (AGENTS.md "Comments"); no setting changes.

- 镜像从 rocm/atom-dev:nightly_202610121516 更新为 nightly_202611101510(ATOM e5b74bc86,ROCm 7.2.4),AToMesh 前端 container_image 同步更新为同一镜像。
- 纯 TP2 [1, 2, 4, 6, 8, 16, 32, 64] 和纯 TP4 [1, 8, 32] 改为纯 TP2 [1, 2, 16, 32]、DTP2 c64、DEP2 [128, 256] 和纯 TP4 [1, 2, 4, 8]。删除 TP2 c4、c6、c8、c64 和 TP4 c32;新增 TP4 c2 和 c4。
- 所有点的 KV cache 从 BF16 改为 MXFP8;index cache 仍为 FP4。
- 去掉 max-num-seqs 128。并发 1 使用 max-num-seqs 2、decode 图 [1, 2] 和 ATOM_PREFILL_DECODE_INTERVAL=1000000,即 ATOM recipes/Kimi-K3.md 的 C1 recipe(ROCm/ATOM#2659)。c256 额外捕获 256。
- 各点设置 ATOM_PREFILL_DECODE_INTERVAL:TP2 c2 和 TP4 c2-c8 为 192,TP2 c16 为 96,TP2 c32 为 40,DP 点为 10。
- DP 点增加 dp-load-balance least_tokens、ATOM_DP_SESSION_AFFINITY=1、ATOM_DP_LB_REQ_EQUIV=512 和 GPU_MAX_HW_QUEUES=5,AIPerf 以 X-Dynamo-Session-Id 发送 session id。
- 所有点增加 timeout-keep-alive 900。
- 新 ATOM 带来 V4.1 DP attention(#2376)、V4.1 MXFP8 KV cache 与 DSpark(#2725)、DP prefill 均衡与图回放(#2551、#2696、#2697)以及默认 HIP Mega-MHC kernel(#2568)。
- DSpark draft 权重精度不变;其 KV cache 随 target 从 BF16 变为 MXFP8。

另将 recipe 和 master config 的头部注释改为行注释(AGENTS.md "Comments"),不改变任何设置。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@yhl-amd yhl-amd added the full-sweep-enabled Full sweep with canary gate; matrix jobs run to completion despite failures label Nov 11, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@yhl-amd yhl-amd changed the title perf(amd): DSv4.1-Flash ATOM AgentX TP4 c8 only on the 1111 nightly / DSv4.1-Flash ATOM AgentX 1111 nightly 只测 TP4 c8 perf(amd): DSv4.1-Flash ATOM AgentX TP4 c1 only on the 1111 nightly / DSv4.1-Flash ATOM AgentX 1111 nightly 只测 TP4 c1 Nov 11, 2026
@yhl-amd yhl-amd added full-sweep-enabled Full sweep with canary gate; matrix jobs run to completion despite failures and removed full-sweep-enabled Full sweep with canary gate; matrix jobs run to completion despite failures labels Nov 11, 2026
@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

yhl-amd and others added 3 commits November 12, 2026 00:25
…1111-tp4c8

# Conflicts:
#	inferencex-e2e/perf-changelog.yaml
…g main

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@yhl-amd yhl-amd added full-sweep-enabled Full sweep with canary gate; matrix jobs run to completion despite failures and removed full-sweep-enabled Full sweep with canary gate; matrix jobs run to completion despite failures labels Nov 13, 2026
…1111-tp4c8

# Conflicts:
#	inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/dsv41flash/atom/mi355x-fp4-mtp/agentic.yaml
#	inferencex-e2e/configs/amd-master.yaml
#	inferencex-e2e/perf-changelog.yaml

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-sweep-enabled Full sweep with canary gate; matrix jobs run to completion despite failures

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants