Repository navigation
[feat] add H3 video configs on H100, H200 and MI355X / [feat] 在 H100、H200 和 MI355X 上新增 H3 视频配置 - #4968
edwingao28 wants to merge 3 commits into
Conversation
Mirror the SGLang H3 deployments (h3-v2.yaml release-v1, C1) with vLLM-Omni: DiT TP2 with Ulysses 1, 2 and 4 on 2, 4 and 8 H100 GPUs, and Ulysses across all GPUs on 1, 2, 4 and 8 H200 or MI355X GPUs. H100 and H200 pin vLLM-Omni v0.32.0; MI355X pins vllm/vllm-omni-rocm:v0.30.0. Co-authored-by: Perplexity <noreply@perplexity.ai>
|
Thanks for the contribution!
中文感谢你的贡献!
|
Co-authored-by: Perplexity <noreply@perplexity.ai>
Co-authored-by: Perplexity <noreply@perplexity.ai>
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=42763188463 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=42763188463 |
|
Superseded by one PR per SKU: #4969 (H100), #4970 (H200), #4971 (MI355X). The sweep started here was cancelled. Created with Perplexity |
There was a problem hiding this comment.
I reviewed this PR and didn't find any bugs. These are new benchmark layouts on CODEOWNER-owned master configs and NVIDIA/AMD recipe paths, and upstream has not measured them yet (TP2 with Ulysses on H100, vLLM-Omni v0.30.0 on MI355X), so a human look is still worthwhile.
What was reviewed:
- H100 recipes: DiT TP2 × USP 1/2/4 equals the 2/4/8 GPU count in the matching search-space rows. Text-encoder TP and VAE patch parallel follow
{gpus_per_node}. - The H200 and MI355X recipes now match the B200 recipe. The only additions are the AITER
FLASH_ATTNbackend andVLLM_ROCM_QUICK_REDUCE_QUANTIZATION: NONEon MI355X. Each one keeps a 40-charVIDEO_RUNTIME_REVISIONthat the binder requires. - The perf-changelog block is appended only at the end, and its
pull/XXXplaceholder follows the CONTRIBUTING.md template. Images are pinned by digest, and the docs agree with the new rows. - Not done: the 11-point sweep on H100, H200 and MI355X. Nothing here shows these layouts actually start up or fit in memory.
Extended reasoning...
The change adds three vLLM-Omni H3 video config keys (H100, H200, MI355X) to nvidia-master.yaml and amd-master.yaml, along with four new or updated srt-slurm recipes, a perf-changelog entry and doc updates. It touches no security-sensitive surface such as auth, secrets or user input handling. I deferred instead of approving because .github/CODEOWNERS assigns specific owners to every changed path (both master configs, the h100/h200/mi355x recipe directories, MODELS.md and video-generation/). The H100 TP2+Ulysses and MI355X v0.30.0 layouts also have no prior runs.
This review covers commit 7da141d, which is no longer the latest commit on this pull request; later commits are not covered by it.
Summary
vLLM-Omni H3 video results exist only on B200 (#4874), while SGLang covers H100, H200, B200 and MI355X. This adds vLLM-Omni on the remaining three SKUs with the same GPU counts, spec and set as each SGLang key (
h3-v2.yamlrelease-v1, C1), so the bound plan hash equals SGLang's (a5c800f1…) on every hardware and the two engines are measured on one workload.h3-bf16-h100-vllm-omni-videovllm/vllm-omni:v0.32.0h3-bf16-h200-vllm-omni-videovllm/vllm-omni:v0.32.0h3-bf16-mi355x-vllm-omni-videovllm/vllm-omni-rocm:v0.30.0FLASH_ATTNChanges:
h3/vllm-omni/: newh100-bf16/(three files named after the SGLang H100 recipes) andmi355x-bf16/; the existingh200-bf16/serving-client.yamlhad no config key and was stale (hard-coded 4 GPUs, older engine revision, no AdaLN setting), so it now matches the B200 recipe. Text-encoder TP and tiled VAE patch parallelism span all GPUs on every point.OMNI_DIFFUSION_PROMPT_EMBED_CACHE=0), cross-request AdaLN projection reuse off (minimax_h3_adaln_cache: false), and the runner's x264 options reach the encoder throughvideo_codec_options.vllm/vllm-omni-rocm:v0.30.0is the newest stable ROCm image; its source layer matches tag v0.30.0 (a8576ccb), which is the pinnedVIDEO_RUNTIME_REVISION, and it supports the same H3 flags (--task-type fl2va,--usp, text-encoder TP, VAE patch parallelism,minimax_h3_adaln_cache, the prompt-embed cache switch). Upstream documents H3 on gfx942/gfx950 in BF16 with theFLASH_ATTN(AITER) backend, which the recipe selects. vLLM quick all-reduce quantization is set toNONEexplicitly per the ROCm quantization rule in MODELS.md.perf-changelog.yamlblock for the three keys; MODELS.md, video-generation.md and SUBMITTING.md list the rows.Risks: MI355X runs vLLM-Omni v0.30.0, not v0.32.0, so the MI355X engine comparison is against an older release. Upstream marks text-encoder TP as not exercised on gfx950 and lists MI355X separately from its MI350 evidence, and TP2 combined with Ulysses on H100 has no upstream H3 measurement; the sweep is the first run of these layouts. Single-GPU H200 runs without offload.
Delivery: needs the PR full sweep on H100, H200 and MI355X (11 points).
中文
概要
vLLM-Omni 的 H3 视频结果目前只有 B200(#4874),而 SGLang 已覆盖 H100、H200、B200 和 MI355X。本 PR 在其余三种 SKU 上新增 vLLM-Omni,GPU 数量、spec 和 set 与各 SGLang key 一致(
h3-v2.yamlrelease-v1,C1),因此每种硬件上绑定的 plan hash 都与 SGLang 相同(a5c800f1…),两种引擎测量同一负载。h3-bf16-h100-vllm-omni-video:vllm/vllm-omni:v0.32.0,2、4、8 GPU,DiT TP2 加 Ulysses 1、2、4(与 SGLang H100 相同)。h3-bf16-h200-vllm-omni-video:vllm/vllm-omni:v0.32.0,1、2、4、8 GPU,Ulysses 覆盖全部 GPU。h3-bf16-mi355x-vllm-omni-video:vllm/vllm-omni-rocm:v0.30.0,1、2、4、8 GPU,Ulysses 覆盖全部 GPU,AITERFLASH_ATTN。改动:
h3/vllm-omni/下的 recipe:新增h100-bf16/(三个文件与 SGLang H100 recipe 同名)和mi355x-bf16/;原有的h200-bf16/serving-client.yaml没有对应的 config key 且已过时(固定 4 GPU、旧引擎版本、缺少 AdaLN 设置),现与 B200 recipe 对齐。所有点位的 text-encoder TP 和 tiled VAE patch 并行都覆盖全部 GPU。video_codec_options传给编码器。vllm/vllm-omni-rocm:v0.30.0是最新的稳定 ROCm 镜像,其源码层与 tag v0.30.0(a8576ccb)一致,并支持相同的 H3 参数。上游文档说明 H3 在 gfx942/gfx950 上以 BF16 和FLASH_ATTN(AITER)后端运行。按 MODELS.md 的 ROCm 量化规则显式设置 vLLM quick all-reduce 量化为NONE。perf-changelog.yaml中为三个 key 新增一个条目;MODELS.md、video-generation.md 和 SUBMITTING.md 已更新。风险: MI355X 使用 vLLM-Omni v0.30.0 而非 v0.32.0。上游标注 gfx950 上未验证 text-encoder TP,H100 上 TP2 与 Ulysses 组合也没有上游 H3 测量;本次 sweep 是这些布局的首次运行。单卡 H200 不使用 offload。
交付: 需要在 H100、H200 和 MI355X 上运行 PR full sweep(11 个点位)。
AI model disclosure
Prepared by an AI agent running in Perplexity Computer (orchestrator model: Claude Opus 5.5, as exposed by the runtime); the delegated agent that wrote the recipes, configs and this description runs in the same session, and its exact model identifier could not be verified.
Created with Perplexity