Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion experimental/video-generation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ the SKU recipe under `benchmarks/single_node/srt-slurm-recipes/h3/` starts the
engine through `python3 -m infx.bench video serve` as a generic service and runs
`python3 -m infx.bench video client` as `benchmark.type: custom`. This directory
is mounted into the job as `/h3-package`. H100, H200 and B200 select
`h3-v0.1.yaml/release-v1`: 20 distinct measured prompts at C1. See
`h3-v0.1.yaml/release-v1`: 20 distinct measured prompts at C1. GB200 retains
`h3.yaml/release-v1` with its separate 107-frame contract. See
[native video delivery](../../inferencex-e2e/docs/video-generation.md) for
artifact identity, reuse validation and post-merge publication.

Expand Down
3 changes: 2 additions & 1 deletion experimental/video-generation/README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@ diffusion 引擎在单节点上提供服务,选定的冻结 YAML 工作负载
`python3 -m infx.bench video serve` 把引擎作为 generic service 启动,
并以 `benchmark.type: custom` 运行 `python3 -m infx.bench video client`。
本目录以 `/h3-package` 挂载进任务。H100、H200 与 B200 选用
`h3-v0.1.yaml/release-v1`,在 C1 测量 20 个不同 prompt。
`h3-v0.1.yaml/release-v1`,在 C1 测量 20 个不同 prompt;GB200 保留
`h3.yaml/release-v1` 及其独立的 107 帧合同。
产物身份、复用校验与合并后发布见
[原生视频交付](../../inferencex-e2e/docs/video-generation.md)。

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
schema: 2
name: h3-sglang-serving-client

resources:
gpu_type: gb200

frontend:
type: none

services:
- name: h3-server
type: generic
nodes: 1
critical: true
start: before_workers
command: [python3, -m, infx.bench, video, serve]
args:
- --model-view
- snapshot
- --
- /opt/sglang/bin/python3
- -m
- evaluator.srt_selected
- serve
- --model-type
- diffusion
- --model-path
- "{model_path}"
- --model-id
- "{model_id}"
- --revision
- "{model_revision}"
- --model-variant
- fl2va
- --num-gpus
- "{gpus_per_node}"
- --ulysses-degree
- "{gpus_per_node}"
- --tp-size
- "1"
- --encoder-parallel
- auto
- --performance-mode
- speed
- --host
- 0.0.0.0
- --port
- "30283"
- --enable-torch-compile
- "false"
- --dit-cpu-offload
- "false"
env:
PYTHONPATH: /h3-package:/infmax-workspace
PYTHONUNBUFFERED: "1"
H3_SERVER_PORT: "30283"
VIDEO_RUNTIME_REVISION: baa9befa381511cd4d56f83980951e5a45fffc6d
HF_HUB_OFFLINE: "1"
TRANSFORMERS_OFFLINE: "1"
readiness:
http:
port: 30283
path: /health
timeout_seconds: 1800
interval_seconds: 5

benchmark:
type: custom
command: python3 -m infx.bench video client --request-seconds 900
env:
PYTHONPATH: /h3-package:/infmax-workspace
PYTHONUNBUFFERED: "1"
H3_SERVER_SERVICE: h3-server
H3_SERVER_PORT: "30283"
23 changes: 23 additions & 0 deletions inferencex-e2e/configs/nvidia-master.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10680,6 +10680,29 @@ h3-bf16-b200-sglang-video:
srt-recipe: serving-client.yaml
conc-list: [1]

h3-bf16-gb200-sglang-video:
image: lmsysorg/sglang:v0.5.22-cu130@sha256:06441be3de19a816ddbac917535e920cbfc7fc43171abc08417394f43634ffb6
model: MiniMaxAI/MiniMax-H3
model-prefix: h3
runner: cluster:gb200-nv
precision: bf16
framework: sglang
multinode: false
srt-recipe-dir: h3/sglang/gb200-bf16
scenarios:
video-generation:
- spec: h3.yaml
set: release-v1
search-space:
- gpus: 2
srt-recipe: serving-client.yaml
conc-list: [1]
- gpus: 4
srt-recipe: serving-client.yaml
conc-list: [1]

# Sol Engine GB200 stays disabled until its arm64 image and serving contract are available.
# h3-bf16-gb200-sol-engine-video:
# Kimi-K3 GB200 disaggregated points: one
# TP16/DCP16 prefill role and one TP16/DCP16 decode role, four 4-GPU nodes each
# (32 GPUs). DSpark depth 4 with the golden acceptance selected by the launcher.
Expand Down
1 change: 1 addition & 0 deletions inferencex-e2e/configs/runners.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -587,6 +587,7 @@ clusters:
MiniMax-M3-MXFP8: {root: lustre-models, dir: MiniMax-M3-MXFP8}
MiniMax-M3-NVFP4: {root: lustre-models, dir: MiniMax-M3-NVFP4}
MiniMax-M3.1-Flash: {root: lustre-models, dir: MiniMax-M3.1-Flash}
MiniMax-H3: {root: hf-hub-cache, dir: models--MiniMaxAI--MiniMax-H3/snapshots/42ed227ee7df40d41602854ae760620d6eb651fe}
Qwen3.5-397B-A17B-FP8: {root: lustre-models, dir: Qwen3.5-397B-A17B-FP8}
Qwen3.5-397B-A17B-NVFP4-V2: {root: lustre-models, dir: Qwen3.5-397B-A17B-NVFP4-V2}
Muse-Spark-Open-816B-A42B-NVFP4: {root: lustre-models, dir: Muse-Spark-Open-816B-A42B-NVFP4}
Expand Down
2 changes: 1 addition & 1 deletion inferencex-e2e/docs/MODELS.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ For non-grandfathered models, `ROCM_QUICK_REDUCE_QUANTIZATION`, `VLLM_ROCM_QUICK
|---|---|---|
| Agentic coding | Long Context, Multi Turn Realistic traffic trace replay with sub agents | Active. This is the trace-replay agentic-coding benchmark (see the srt-slurm recipes under [`benchmarks/single_node/srt-slurm-recipes/`](../benchmarks/single_node/srt-slurm-recipes) and the shared client [`benchmarks/srt_agentic.sh`](../benchmarks/srt_agentic.sh)). Going forward, new models will likely be onboarded with agentic coding only. Speculative decoding may be enabled or disabled to produce the best Pareto points; a separate non-spec-decode A/B baseline is not required (see [Deprecation policy](#deprecation-policy)). |
| Single-turn 8k1k | 8192 / 1024 | Active. This is the primary fixed-sequence-length scenario. |
| Video generation | H3 v0.1: 124 frames at 24 fps, 1344x768, C1 | MiniMax-H3 uses native srt-slurm: H100/H200/B200 SGLang and H200 vLLM-Omni select four-GPU, 20-clip `h3-v0.1.yaml` C1 points. The client and server live in [`experimental/video-generation`](../../experimental/video-generation); energy is reported per valid clip. Config presence does not qualify execution, power, or quality. AMD remains outside active configs. See [video delivery](video-generation.md). |
| Video generation | H3 v0.1: 124 frames at 24 fps, 1344x768, C1; legacy GB200: 107 frames | MiniMax-H3 uses native srt-slurm: H100/H200/B200 SGLang and H200 vLLM-Omni select four-GPU, 20-clip `h3-v0.1.yaml` C1 points. GB200 SGLang retains separate two- and four-GPU `h3.yaml` `release-v1` C1 points under the old contract. The client and server live in [`experimental/video-generation`](../../experimental/video-generation); energy is reported per valid clip. Config presence does not qualify execution, power, or quality. AMD remains outside active configs. See [video delivery](video-generation.md). |
| Single-turn 1k1k | 1024 / 1024 | Deprecated since 2026-07-17 ([#2263](https://github.com/SemiAnalysisAI/InferenceX/pull/2263)), to save GPU cluster time for higher-priority real-world agentic-coding benchmarks and new frontier models. Archived configs lived in `configs/deprecated/` until it was deleted (#3463); see git history. The GLM-5.1 B200 TileRT point added later in [#2533](https://github.com/SemiAnalysisAI/InferenceX/pull/2533) was the last 1k1k config; it was retired on 2026-10-01 ([#3638](https://github.com/SemiAnalysisAI/InferenceX/pull/3638)). |
| Single-turn 1k8k | 1024 / 8192 | **Deprecated for all models** since 2026-03-27 ([#911](https://github.com/SemiAnalysisAI/InferenceX/pull/911)), to save GPU cluster time for higher-priority real-world agentic-coding benchmarks and new frontier models. Configs were removed, not archived. |

Expand Down
30 changes: 25 additions & 5 deletions inferencex-e2e/docs/video-generation.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,33 @@ The `video-generation` scenario schedules one server job per deployment. H100,
H200, and B200 SGLang and H200 vLLM-Omni select four-GPU C1 deployments using
`h3-v0.1.yaml`: 20 distinct measured prompts, a separate warmup, 49 denoising
updates, and 124 frames at 24 fps and 1344×768. A five-second request produces
124/24 seconds of video. Config presence is not GPU or quality qualification.
The H100/H200/B200 SGLang recipes pin the public amd64 v0.5.23-cu130 image.
Actual step execution, media,
124/24 seconds of video. GB200 retains its separate two- and four-GPU C1
deployments on `h3.yaml` `release-v1`: 20 measured clips and one warmup at 50
updates and 107 frames. The two workloads are not directly comparable. Config
presence is not GPU or quality qualification.
The H100/H200/B200 SGLang recipes pin the public amd64 v0.5.23-cu130 image;
GB200 retains its arm64 v0.5.22-cu130 image. Actual step execution, media,
and power must be checked in the original run. The YAML workload has no `model.path`
or Python field: the binder derives the pinned model snapshot path from `MODEL`,
`HF_HUB_CACHE`, and the model revision. The service recipe selects the engine executable.
Native power is accepted only when its sampled GPU UUIDs match the server's
`spec.json` and `launch.json` assignment.
The GB200 checkpoint must be staged at the pinned snapshot under the cluster's
shared HF cache before dispatch. The `gb200-nv.models.entries` record points to
that snapshot; it does not download it. On the authorized submit host, set
`HF_HUB_CACHE` to the host-side shared cache path from `configs/runners.yaml`, then stage it with:

```bash
hf download MiniMaxAI/MiniMax-H3 \
--revision 42ed227ee7df40d41602854ae760620d6eb651fe \
--cache-dir "$HF_HUB_CACHE"
```

Verify the returned snapshot revision and required FL2VA and model-index files
before dispatch.
Sol Engine remains outside the executable matrix until a public arm64 image and
its actual serve command, readiness endpoint, request payload, and media response
contract are available.
The H100 runner provides an HF hub cache mount, but the pinned H3 snapshot must
be verified there before execution. MI355X is excluded because the prepared
CUDA image cannot run on AMD. Historical artifacts remain evidence of their
Expand Down Expand Up @@ -98,6 +117,7 @@ adapters and workloads; the table does not establish final-head GPU qualificatio
| `h3-bf16-h200-sglang-video` | `h3/sglang/h200-bf16/serving-client.yaml` | 4 | 20 at C1, v0.1 |
| `h3-bf16-h200-vllm-omni-video` | `h3/vllm-omni/h200-bf16/serving-client.yaml` | 4 | 20 at C1, v0.1 |
| `h3-bf16-b200-sglang-video` | `h3/sglang/b200-bf16/serving-client.yaml` | 4 | 20 at C1, v0.1 |
| `h3-bf16-gb200-sglang-video` | `h3/sglang/gb200-bf16/serving-client.yaml` | 2 and 4 | 20 at C1 per deployment |

1. Choose an existing combination and keep its frozen workload, model revision,
concurrency population and measurement boundary. Put the full engine command
Expand All @@ -108,8 +128,8 @@ adapters and workloads; the table does not establish final-head GPU qualificatio
2. Change the existing row in `configs/nvidia-master.yaml`, or copy it with a
distinct key and recipe for a deployment that must remain separately selectable.
Pin the image by digest and use its real interpreter and architecture. The
H100/H200/B200 SGLang rows use the public amd64 image. A shared image does
not qualify a different topology or SKU: validate
H200/B200 SGLang rows use the public amd64 image, while GB200 uses its arm64
digest. A shared image does not qualify a different topology or SKU: validate
the selected deployments through their sweep. Maintainers stage model snapshots in the runner's
existing HF cache; contributors do not patch the cache or SSH extra launch flags.
3. Render locally from `inferencex-e2e/`, using the repository's pinned srt-slurm
Expand Down
7 changes: 7 additions & 0 deletions inferencex-e2e/perf-changelog.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11604,3 +11604,10 @@
- "Select 20 distinct measured prompts, a disjoint warmup, 124 frames, 49 denoising updates, pinned images, and explicit cross-request conditioning-cache policy."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2916

- config-keys:
- h3-bf16-gb200-sglang-video
scenario-type: [video-generation]
description:
- "Add two- and four-GPU MiniMax-H3 SGLang deployments on GB200 with the pinned public arm64 v0.5.22-cu130 image and model snapshot."
- "Select the shared release-v1 C1 workload: 20 measured clips and one warmup, 50 inference steps, 107 frames at 24 fps and 1344x768."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4644
Loading