Skip to content

[feat] add H3 video configs on H100, H200 and MI355X / [feat] 在 H100、H200 和 MI355X 上新增 H3 视频配置 - #4968

Closed
edwingao28 wants to merge 3 commits into
mainfrom
feat/h3-vllm-omni-h100-h200-mi355x
Closed

edwingao28 wants to merge 3 commits into
mainfrom
feat/h3-vllm-omni-h100-h200-mi355x

Conversation

@edwingao28

Copy link
Copy Markdown
Collaborator

Summary

vLLM-Omni H3 video results exist only on B200 (#4874), while SGLang covers H100, H200, B200 and MI355X. This adds vLLM-Omni on the remaining three SKUs with the same GPU counts, spec and set as each SGLang key (h3-v2.yaml release-v1, C1), so the bound plan hash equals SGLang's (a5c800f1…) on every hardware and the two engines are measured on one workload.

Key Image (pinned by linux/amd64 digest) GPUs Layout
h3-bf16-h100-vllm-omni-video vllm/vllm-omni:v0.32.0 2, 4, 8 DiT TP2 with Ulysses 1, 2, 4 (as SGLang H100)
h3-bf16-h200-vllm-omni-video vllm/vllm-omni:v0.32.0 1, 2, 4, 8 Ulysses across all GPUs
h3-bf16-mi355x-vllm-omni-video vllm/vllm-omni-rocm:v0.30.0 1, 2, 4, 8 Ulysses across all GPUs, AITER FLASH_ATTN

Changes:

  • Recipes under h3/vllm-omni/: new h100-bf16/ (three files named after the SGLang H100 recipes) and mi355x-bf16/; the existing h200-bf16/serving-client.yaml had no config key and was stale (hard-coded 4 GPUs, older engine revision, no AdaLN setting), so it now matches the B200 recipe. Text-encoder TP and tiled VAE patch parallelism span all GPUs on every point.
  • Same work across requests, unchanged from [VideoGenX] add vLLM-Omni H3 video config on B200 / [VideoGenX] 在 B200 上新增 vLLM-Omni H3 视频配置 #4874: prompt-embedding cache off (OMNI_DIFFUSION_PROMPT_EMBED_CACHE=0), cross-request AdaLN projection reuse off (minimax_h3_adaln_cache: false), and the runner's x264 options reach the encoder through video_codec_options.
  • H100: H3 does not fit one 80 GB GPU and pure Ulysses replicates the DiT on every rank, so H100 uses DiT TP2 like SGLang.
  • MI355X: vLLM-Omni has no v0.32.0 ROCm image yet. vllm/vllm-omni-rocm:v0.30.0 is the newest stable ROCm image; its source layer matches tag v0.30.0 (a8576ccb), which is the pinned VIDEO_RUNTIME_REVISION, and it supports the same H3 flags (--task-type fl2va, --usp, text-encoder TP, VAE patch parallelism, minimax_h3_adaln_cache, the prompt-embed cache switch). Upstream documents H3 on gfx942/gfx950 in BF16 with the FLASH_ATTN (AITER) backend, which the recipe selects. vLLM quick all-reduce quantization is set to NONE explicitly per the ROCm quantization rule in MODELS.md.
  • One perf-changelog.yaml block for the three keys; MODELS.md, video-generation.md and SUBMITTING.md list the rows.

Risks: MI355X runs vLLM-Omni v0.30.0, not v0.32.0, so the MI355X engine comparison is against an older release. Upstream marks text-encoder TP as not exercised on gfx950 and lists MI355X separately from its MI350 evidence, and TP2 combined with Ulysses on H100 has no upstream H3 measurement; the sweep is the first run of these layouts. Single-GPU H200 runs without offload.

Delivery: needs the PR full sweep on H100, H200 and MI355X (11 points).

中文

概要

vLLM-Omni 的 H3 视频结果目前只有 B200(#4874),而 SGLang 已覆盖 H100、H200、B200 和 MI355X。本 PR 在其余三种 SKU 上新增 vLLM-Omni,GPU 数量、spec 和 set 与各 SGLang key 一致(h3-v2.yaml release-v1,C1),因此每种硬件上绑定的 plan hash 都与 SGLang 相同(a5c800f1…),两种引擎测量同一负载。

  • h3-bf16-h100-vllm-omni-video:vllm/vllm-omni:v0.32.0,2、4、8 GPU,DiT TP2 加 Ulysses 1、2、4(与 SGLang H100 相同)。
  • h3-bf16-h200-vllm-omni-video:vllm/vllm-omni:v0.32.0,1、2、4、8 GPU,Ulysses 覆盖全部 GPU。
  • h3-bf16-mi355x-vllm-omni-video:vllm/vllm-omni-rocm:v0.30.0,1、2、4、8 GPU,Ulysses 覆盖全部 GPU,AITER FLASH_ATTN。

改动:

  • h3/vllm-omni/ 下的 recipe:新增 h100-bf16/(三个文件与 SGLang H100 recipe 同名)和 mi355x-bf16/;原有的 h200-bf16/serving-client.yaml 没有对应的 config key 且已过时(固定 4 GPU、旧引擎版本、缺少 AdaLN 设置),现与 B200 recipe 对齐。所有点位的 text-encoder TP 和 tiled VAE patch 并行都覆盖全部 GPU。
  • 与 [VideoGenX] add vLLM-Omni H3 video config on B200 / [VideoGenX] 在 B200 上新增 vLLM-Omni H3 视频配置 #4874 相同的同工作量设置:关闭 prompt-embedding 缓存,关闭跨请求 AdaLN 投影复用,runner 的 x264 参数通过 video_codec_options 传给编码器。
  • H100:H3 无法放入单张 80 GB GPU,纯 Ulysses 会在每个 rank 复制 DiT,因此与 SGLang 一样使用 DiT TP2。
  • MI355X:vLLM-Omni 尚无 v0.32.0 ROCm 镜像。vllm/vllm-omni-rocm:v0.30.0 是最新的稳定 ROCm 镜像,其源码层与 tag v0.30.0(a8576ccb)一致,并支持相同的 H3 参数。上游文档说明 H3 在 gfx942/gfx950 上以 BF16 和 FLASH_ATTN(AITER)后端运行。按 MODELS.md 的 ROCm 量化规则显式设置 vLLM quick all-reduce 量化为 NONE。
  • perf-changelog.yaml 中为三个 key 新增一个条目;MODELS.md、video-generation.md 和 SUBMITTING.md 已更新。

风险: MI355X 使用 vLLM-Omni v0.30.0 而非 v0.32.0。上游标注 gfx950 上未验证 text-encoder TP,H100 上 TP2 与 Ulysses 组合也没有上游 H3 测量;本次 sweep 是这些布局的首次运行。单卡 H200 不使用 offload。

交付: 需要在 H100、H200 和 MI355X 上运行 PR full sweep(11 个点位)。

AI model disclosure

Prepared by an AI agent running in Perplexity Computer (orchestrator model: Claude Opus 5.5, as exposed by the runtime); the delegated agent that wrote the recipes, configs and this description runs in the same session, and its exact model identifier could not be verified.

Created with Perplexity

Mirror the SGLang H3 deployments (h3-v2.yaml release-v1, C1) with
vLLM-Omni: DiT TP2 with Ulysses 1, 2 and 4 on 2, 4 and 8 H100 GPUs, and
Ulysses across all GPUs on 1, 2, 4 and 8 H200 or MI355X GPUs. H100 and
H200 pin vLLM-Omni v0.32.0; MI355X pins vllm/vllm-omni-rocm:v0.30.0.

Co-authored-by: Perplexity <noreply@perplexity.ai>
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

Co-authored-by: Perplexity <noreply@perplexity.ai>
@edwingao28 edwingao28 added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Nov 10, 2026
Co-authored-by: Perplexity <noreply@perplexity.ai>
@edwingao28 edwingao28 removed the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Nov 10, 2026
@github-actions

Copy link
Copy Markdown
Contributor

@edwingao28

Copy link
Copy Markdown
Collaborator Author

Superseded by one PR per SKU: #4969 (H100), #4970 (H200), #4971 (MI355X). The sweep started here was cancelled.

Created with Perplexity

@edwingao28 edwingao28 closed this Nov 10, 2026
@edwingao28 edwingao28 changed the title [VideoGenX] add vLLM-Omni H3 video configs on H100, H200 and MI355X / [VideoGenX] 在 H100、H200 和 MI355X 上新增 vLLM-Omni H3 视频配置 [feat] add H3 video configs on H100, H200 and MI355X / [feat] 在 H100、H200 和 MI355X 上新增 H3 视频配置 Nov 10, 2026

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and didn't find any bugs. These are new benchmark layouts on CODEOWNER-owned master configs and NVIDIA/AMD recipe paths, and upstream has not measured them yet (TP2 with Ulysses on H100, vLLM-Omni v0.30.0 on MI355X), so a human look is still worthwhile.

What was reviewed:

  • H100 recipes: DiT TP2 × USP 1/2/4 equals the 2/4/8 GPU count in the matching search-space rows. Text-encoder TP and VAE patch parallel follow {gpus_per_node}.
  • The H200 and MI355X recipes now match the B200 recipe. The only additions are the AITER FLASH_ATTN backend and VLLM_ROCM_QUICK_REDUCE_QUANTIZATION: NONE on MI355X. Each one keeps a 40-char VIDEO_RUNTIME_REVISION that the binder requires.
  • The perf-changelog block is appended only at the end, and its pull/XXX placeholder follows the CONTRIBUTING.md template. Images are pinned by digest, and the docs agree with the new rows.
  • Not done: the 11-point sweep on H100, H200 and MI355X. Nothing here shows these layouts actually start up or fit in memory.
Extended reasoning...

The change adds three vLLM-Omni H3 video config keys (H100, H200, MI355X) to nvidia-master.yaml and amd-master.yaml, along with four new or updated srt-slurm recipes, a perf-changelog entry and doc updates. It touches no security-sensitive surface such as auth, secrets or user input handling. I deferred instead of approving because .github/CODEOWNERS assigns specific owners to every changed path (both master configs, the h100/h200/mi355x recipe directories, MODELS.md and video-generation/). The H100 TP2+Ulysses and MI355X v0.30.0 layouts also have no prior runs.

This review covers commit 7da141d, which is no longer the latest commit on this pull request; later commits are not covered by it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant