diff --git a/experimental/video-generation/README.md b/experimental/video-generation/README.md index baaba14eb9..62ba7ad276 100644 --- a/experimental/video-generation/README.md +++ b/experimental/video-generation/README.md @@ -16,7 +16,8 @@ the SKU recipe under `benchmarks/single_node/srt-slurm-recipes/h3/` starts the engine through `python3 -m infx.bench video serve` as a generic service and runs `python3 -m infx.bench video client` as `benchmark.type: custom`. This directory is mounted into the job as `/h3-package`. H100, H200 and B200 select -`h3-v0.1.yaml/release-v1`: 20 distinct measured prompts at C1. See +`h3-v0.1.yaml/release-v1`: 20 distinct measured prompts at C1. GB200 retains +`h3.yaml/release-v1` with its separate 107-frame contract. See [native video delivery](../../inferencex-e2e/docs/video-generation.md) for artifact identity, reuse validation and post-merge publication. diff --git a/experimental/video-generation/README_zh.md b/experimental/video-generation/README_zh.md index b803411438..3ae11e39c2 100644 --- a/experimental/video-generation/README_zh.md +++ b/experimental/video-generation/README_zh.md @@ -14,7 +14,8 @@ diffusion 引擎在单节点上提供服务,选定的冻结 YAML 工作负载 `python3 -m infx.bench video serve` 把引擎作为 generic service 启动, 并以 `benchmark.type: custom` 运行 `python3 -m infx.bench video client`。 本目录以 `/h3-package` 挂载进任务。H100、H200 与 B200 选用 -`h3-v0.1.yaml/release-v1`,在 C1 测量 20 个不同 prompt。 +`h3-v0.1.yaml/release-v1`,在 C1 测量 20 个不同 prompt;GB200 保留 +`h3.yaml/release-v1` 及其独立的 107 帧合同。 产物身份、复用校验与合并后发布见 [原生视频交付](../../inferencex-e2e/docs/video-generation.md)。 diff --git a/inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/h3/sglang/gb200-bf16/serving-client.yaml b/inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/h3/sglang/gb200-bf16/serving-client.yaml new file mode 100644 index 0000000000..c18f69c2b6 --- /dev/null +++ b/inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/h3/sglang/gb200-bf16/serving-client.yaml @@ -0,0 +1,74 @@ +schema: 2 +name: h3-sglang-serving-client + +resources: + gpu_type: gb200 + +frontend: + type: none + +services: + - name: h3-server + type: generic + nodes: 1 + critical: true + start: before_workers + command: [python3, -m, infx.bench, video, serve] + args: + - --model-view + - snapshot + - -- + - /opt/sglang/bin/python3 + - -m + - evaluator.srt_selected + - serve + - --model-type + - diffusion + - --model-path + - "{model_path}" + - --model-id + - "{model_id}" + - --revision + - "{model_revision}" + - --model-variant + - fl2va + - --num-gpus + - "{gpus_per_node}" + - --ulysses-degree + - "{gpus_per_node}" + - --tp-size + - "1" + - --encoder-parallel + - auto + - --performance-mode + - speed + - --host + - 0.0.0.0 + - --port + - "30283" + - --enable-torch-compile + - "false" + - --dit-cpu-offload + - "false" + env: + PYTHONPATH: /h3-package:/infmax-workspace + PYTHONUNBUFFERED: "1" + H3_SERVER_PORT: "30283" + VIDEO_RUNTIME_REVISION: baa9befa381511cd4d56f83980951e5a45fffc6d + HF_HUB_OFFLINE: "1" + TRANSFORMERS_OFFLINE: "1" + readiness: + http: + port: 30283 + path: /health + timeout_seconds: 1800 + interval_seconds: 5 + +benchmark: + type: custom + command: python3 -m infx.bench video client --request-seconds 900 + env: + PYTHONPATH: /h3-package:/infmax-workspace + PYTHONUNBUFFERED: "1" + H3_SERVER_SERVICE: h3-server + H3_SERVER_PORT: "30283" diff --git a/inferencex-e2e/configs/nvidia-master.yaml b/inferencex-e2e/configs/nvidia-master.yaml index e983c5f83a..efbb3fcaea 100644 --- a/inferencex-e2e/configs/nvidia-master.yaml +++ b/inferencex-e2e/configs/nvidia-master.yaml @@ -10680,6 +10680,29 @@ h3-bf16-b200-sglang-video: srt-recipe: serving-client.yaml conc-list: [1] +h3-bf16-gb200-sglang-video: + image: lmsysorg/sglang:v0.5.22-cu130@sha256:06441be3de19a816ddbac917535e920cbfc7fc43171abc08417394f43634ffb6 + model: MiniMaxAI/MiniMax-H3 + model-prefix: h3 + runner: cluster:gb200-nv + precision: bf16 + framework: sglang + multinode: false + srt-recipe-dir: h3/sglang/gb200-bf16 + scenarios: + video-generation: + - spec: h3.yaml + set: release-v1 + search-space: + - gpus: 2 + srt-recipe: serving-client.yaml + conc-list: [1] + - gpus: 4 + srt-recipe: serving-client.yaml + conc-list: [1] + +# Sol Engine GB200 stays disabled until its arm64 image and serving contract are available. +# h3-bf16-gb200-sol-engine-video: # Kimi-K3 GB200 disaggregated points: one # TP16/DCP16 prefill role and one TP16/DCP16 decode role, four 4-GPU nodes each # (32 GPUs). DSpark depth 4 with the golden acceptance selected by the launcher. diff --git a/inferencex-e2e/configs/runners.yaml b/inferencex-e2e/configs/runners.yaml index 9243e6eda5..0b2aba97d8 100644 --- a/inferencex-e2e/configs/runners.yaml +++ b/inferencex-e2e/configs/runners.yaml @@ -587,6 +587,7 @@ clusters: MiniMax-M3-MXFP8: {root: lustre-models, dir: MiniMax-M3-MXFP8} MiniMax-M3-NVFP4: {root: lustre-models, dir: MiniMax-M3-NVFP4} MiniMax-M3.1-Flash: {root: lustre-models, dir: MiniMax-M3.1-Flash} + MiniMax-H3: {root: hf-hub-cache, dir: models--MiniMaxAI--MiniMax-H3/snapshots/42ed227ee7df40d41602854ae760620d6eb651fe} Qwen3.5-397B-A17B-FP8: {root: lustre-models, dir: Qwen3.5-397B-A17B-FP8} Qwen3.5-397B-A17B-NVFP4-V2: {root: lustre-models, dir: Qwen3.5-397B-A17B-NVFP4-V2} Muse-Spark-Open-816B-A42B-NVFP4: {root: lustre-models, dir: Muse-Spark-Open-816B-A42B-NVFP4} diff --git a/inferencex-e2e/docs/MODELS.md b/inferencex-e2e/docs/MODELS.md index ea539992c6..35e732e120 100644 --- a/inferencex-e2e/docs/MODELS.md +++ b/inferencex-e2e/docs/MODELS.md @@ -36,7 +36,7 @@ For non-grandfathered models, `ROCM_QUICK_REDUCE_QUANTIZATION`, `VLLM_ROCM_QUICK |---|---|---| | Agentic coding | Long Context, Multi Turn Realistic traffic trace replay with sub agents | Active. This is the trace-replay agentic-coding benchmark (see the srt-slurm recipes under [`benchmarks/single_node/srt-slurm-recipes/`](../benchmarks/single_node/srt-slurm-recipes) and the shared client [`benchmarks/srt_agentic.sh`](../benchmarks/srt_agentic.sh)). Going forward, new models will likely be onboarded with agentic coding only. Speculative decoding may be enabled or disabled to produce the best Pareto points; a separate non-spec-decode A/B baseline is not required (see [Deprecation policy](#deprecation-policy)). | | Single-turn 8k1k | 8192 / 1024 | Active. This is the primary fixed-sequence-length scenario. | -| Video generation | H3 v0.1: 124 frames at 24 fps, 1344x768, C1 | MiniMax-H3 uses native srt-slurm: H100/H200/B200 SGLang and H200 vLLM-Omni select four-GPU, 20-clip `h3-v0.1.yaml` C1 points. The client and server live in [`experimental/video-generation`](../../experimental/video-generation); energy is reported per valid clip. Config presence does not qualify execution, power, or quality. AMD remains outside active configs. See [video delivery](video-generation.md). | +| Video generation | H3 v0.1: 124 frames at 24 fps, 1344x768, C1; legacy GB200: 107 frames | MiniMax-H3 uses native srt-slurm: H100/H200/B200 SGLang and H200 vLLM-Omni select four-GPU, 20-clip `h3-v0.1.yaml` C1 points. GB200 SGLang retains separate two- and four-GPU `h3.yaml` `release-v1` C1 points under the old contract. The client and server live in [`experimental/video-generation`](../../experimental/video-generation); energy is reported per valid clip. Config presence does not qualify execution, power, or quality. AMD remains outside active configs. See [video delivery](video-generation.md). | | Single-turn 1k1k | 1024 / 1024 | Deprecated since 2026-07-17 ([#2263](https://github.com/SemiAnalysisAI/InferenceX/pull/2263)), to save GPU cluster time for higher-priority real-world agentic-coding benchmarks and new frontier models. Archived configs lived in `configs/deprecated/` until it was deleted (#3463); see git history. The GLM-5.1 B200 TileRT point added later in [#2533](https://github.com/SemiAnalysisAI/InferenceX/pull/2533) was the last 1k1k config; it was retired on 2026-10-01 ([#3638](https://github.com/SemiAnalysisAI/InferenceX/pull/3638)). | | Single-turn 1k8k | 1024 / 8192 | **Deprecated for all models** since 2026-03-27 ([#911](https://github.com/SemiAnalysisAI/InferenceX/pull/911)), to save GPU cluster time for higher-priority real-world agentic-coding benchmarks and new frontier models. Configs were removed, not archived. | diff --git a/inferencex-e2e/docs/video-generation.md b/inferencex-e2e/docs/video-generation.md index 16565da329..7d89ff1b27 100644 --- a/inferencex-e2e/docs/video-generation.md +++ b/inferencex-e2e/docs/video-generation.md @@ -4,14 +4,33 @@ The `video-generation` scenario schedules one server job per deployment. H100, H200, and B200 SGLang and H200 vLLM-Omni select four-GPU C1 deployments using `h3-v0.1.yaml`: 20 distinct measured prompts, a separate warmup, 49 denoising updates, and 124 frames at 24 fps and 1344×768. A five-second request produces -124/24 seconds of video. Config presence is not GPU or quality qualification. -The H100/H200/B200 SGLang recipes pin the public amd64 v0.5.23-cu130 image. -Actual step execution, media, +124/24 seconds of video. GB200 retains its separate two- and four-GPU C1 +deployments on `h3.yaml` `release-v1`: 20 measured clips and one warmup at 50 +updates and 107 frames. The two workloads are not directly comparable. Config +presence is not GPU or quality qualification. +The H100/H200/B200 SGLang recipes pin the public amd64 v0.5.23-cu130 image; +GB200 retains its arm64 v0.5.22-cu130 image. Actual step execution, media, and power must be checked in the original run. The YAML workload has no `model.path` or Python field: the binder derives the pinned model snapshot path from `MODEL`, `HF_HUB_CACHE`, and the model revision. The service recipe selects the engine executable. Native power is accepted only when its sampled GPU UUIDs match the server's `spec.json` and `launch.json` assignment. +The GB200 checkpoint must be staged at the pinned snapshot under the cluster's +shared HF cache before dispatch. The `gb200-nv.models.entries` record points to +that snapshot; it does not download it. On the authorized submit host, set +`HF_HUB_CACHE` to the host-side shared cache path from `configs/runners.yaml`, then stage it with: + +```bash +hf download MiniMaxAI/MiniMax-H3 \ + --revision 42ed227ee7df40d41602854ae760620d6eb651fe \ + --cache-dir "$HF_HUB_CACHE" +``` + +Verify the returned snapshot revision and required FL2VA and model-index files +before dispatch. +Sol Engine remains outside the executable matrix until a public arm64 image and +its actual serve command, readiness endpoint, request payload, and media response +contract are available. The H100 runner provides an HF hub cache mount, but the pinned H3 snapshot must be verified there before execution. MI355X is excluded because the prepared CUDA image cannot run on AMD. Historical artifacts remain evidence of their @@ -98,6 +117,7 @@ adapters and workloads; the table does not establish final-head GPU qualificatio | `h3-bf16-h200-sglang-video` | `h3/sglang/h200-bf16/serving-client.yaml` | 4 | 20 at C1, v0.1 | | `h3-bf16-h200-vllm-omni-video` | `h3/vllm-omni/h200-bf16/serving-client.yaml` | 4 | 20 at C1, v0.1 | | `h3-bf16-b200-sglang-video` | `h3/sglang/b200-bf16/serving-client.yaml` | 4 | 20 at C1, v0.1 | +| `h3-bf16-gb200-sglang-video` | `h3/sglang/gb200-bf16/serving-client.yaml` | 2 and 4 | 20 at C1 per deployment | 1. Choose an existing combination and keep its frozen workload, model revision, concurrency population and measurement boundary. Put the full engine command @@ -108,8 +128,8 @@ adapters and workloads; the table does not establish final-head GPU qualificatio 2. Change the existing row in `configs/nvidia-master.yaml`, or copy it with a distinct key and recipe for a deployment that must remain separately selectable. Pin the image by digest and use its real interpreter and architecture. The - H100/H200/B200 SGLang rows use the public amd64 image. A shared image does - not qualify a different topology or SKU: validate + H200/B200 SGLang rows use the public amd64 image, while GB200 uses its arm64 + digest. A shared image does not qualify a different topology or SKU: validate the selected deployments through their sweep. Maintainers stage model snapshots in the runner's existing HF cache; contributors do not patch the cache or SSH extra launch flags. 3. Render locally from `inferencex-e2e/`, using the repository's pinned srt-slurm diff --git a/inferencex-e2e/perf-changelog.yaml b/inferencex-e2e/perf-changelog.yaml index 88fce5efe2..af7092c734 100644 --- a/inferencex-e2e/perf-changelog.yaml +++ b/inferencex-e2e/perf-changelog.yaml @@ -11604,3 +11604,10 @@ - "Select 20 distinct measured prompts, a disjoint warmup, 124 frames, 49 denoising updates, pinned images, and explicit cross-request conditioning-cache policy." pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2916 +- config-keys: + - h3-bf16-gb200-sglang-video + scenario-type: [video-generation] + description: + - "Add two- and four-GPU MiniMax-H3 SGLang deployments on GB200 with the pinned public arm64 v0.5.22-cu130 image and model snapshot." + - "Select the shared release-v1 C1 workload: 20 measured clips and one warmup, 50 inference steps, 107 frames at 24 fps and 1344x768." + pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4644