Skip to content

feat(launch): allow vanilla SGLang and vLLM routers on the GB200 multi-node lane - #4812

Merged
cquil11 merged 2 commits into
mainfrom
cam/gb200-lane-vanilla-routers
Nov 5, 2026
Merged

cquil11 merged 2 commits into
mainfrom
cam/gb200-lane-vanilla-routers

Conversation

@cquil11

@cquil11 cquil11 commented Nov 5, 2026

Copy link
Copy Markdown
Collaborator

The gb200-nv multi-node srt-slurm lane only accepted Dynamo and llm-d frameworks, so recipes that front workers with srt-slurm's sglang-router or vllm-router frontend (framework sglang / vllm) were refused before setup. GB300 already accepts any framework.

This adds sglang and vllm to the gb200-nv SRT_MULTI allowlist so the Kimi K3.1 GB200 SGLang and vLLM PRs can run with the vanilla routers instead of Dynamo, whose K3.1 build (ai-dynamo/dynamo#18124) does not compile.

@cquil11
cquil11 requested a review from a team November 5, 2026 01:14
@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

inferencex-e2e/infx diff: +3 -1

Generally most InferenceX submissions don't need to add changes to the infx package. On the off chance they do, generally the diff is small. Please prompt your agent to clean your PR to keep your diff to only necessary & minimal changes too. We have an increased code quality bar for changes to the infx package.

中文

大多数 InferenceX 提交通常不需要修改 infx 包。即使确实需要,改动通常也很小。请让你的 agent 清理 PR,使改动只包含必要且最小的内容。infx 包的改动有更高的代码质量要求。

("b300-dsxe", LaunchPath.SRT_MULTI): SrtLane(frameworks=_DYNAMO),
("gb200-nv", LaunchPath.SRT_MULTI): SrtLane(
frameworks=any_of("dynamo-sglang", "dynamo-trt", "dynamo-vllm", "llmd-vllm"),
frameworks=any_of("dynamo-sglang", "dynamo-trt", "dynamo-vllm", "llmd-vllm", "sglang", "vllm"),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 The CI Lint job will fail on this PR: the new frameworks= line is 103 characters, over the 100-character format limit. ruff format --check rejects it because line-length = 100 in infx/ruff.toml. E501 is ignored, so ruff check passes, but the formatter check does not. Fix: run ruff format inferencex-e2e/infx so the formatter wraps the any_of(...) arguments. One argument per line with a trailing comma, matching the eval_unsets tuple at lines 80-83, is the result.

Why this was flagged

The changed line 54 in inferencex-e2e/infx/launch/drivers/srt/lanes.py is 8 spaces of indent plus frameworks=any_of("dynamo-sglang", "dynamo-trt", "dynamo-vllm", "llmd-vllm", "sglang", "vllm"),, which comes to 103 columns. inferencex-e2e/infx/ruff.toml:3 sets line-length = 100. .github/workflows/ci.yml:60 runs ruff@ latest format --check --diff inferencex-e2e/infx. The formatter would split this call across lines, so the check prints a diff and exits non-zero. On the base branch the line was 89 columns and the check passed. The E501 entry in the ruff.toml ignore list only silences the linter rule. It does not apply to ruff format --check, so it does not stop the failure. AGENTS.md also requires Ruff-formatted Python under infx/.

Verification: Line 54 of inferencex-e2e/infx/launch/drivers/srt/lanes.py is now 103 columns. inferencex-e2e/infx/ruff.toml:3 sets line-length = 100. E501 is in the lint ignore list (ruff.toml:10), so ruff check passes. But .github/workflows/ci.yml:60 runs ruff@ latest format --check --diff inferencex-e2e/infx, and the Lint matrix entry fails. On the base, the line was 85 columns and passed.

@cquil11
cquil11 merged commit 74ff0cd into main Nov 5, 2026
3 checks passed
@cquil11
cquil11 deleted the cam/gb200-lane-vanilla-routers branch November 5, 2026 01:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant