Repository navigation
feat(launch): allow vanilla SGLang and vLLM routers on the GB200 multi-node lane - #4812
Conversation
|
Generally most InferenceX submissions don't need to add changes to the infx package. On the off chance they do, generally the diff is small. Please prompt your agent to clean your PR to keep your diff to only necessary & minimal changes too. We have an increased code quality bar for changes to the infx package. 中文大多数 InferenceX 提交通常不需要修改 infx 包。即使确实需要,改动通常也很小。请让你的 agent 清理 PR,使改动只包含必要且最小的内容。infx 包的改动有更高的代码质量要求。 |
| ("b300-dsxe", LaunchPath.SRT_MULTI): SrtLane(frameworks=_DYNAMO), | ||
| ("gb200-nv", LaunchPath.SRT_MULTI): SrtLane( | ||
| frameworks=any_of("dynamo-sglang", "dynamo-trt", "dynamo-vllm", "llmd-vllm"), | ||
| frameworks=any_of("dynamo-sglang", "dynamo-trt", "dynamo-vllm", "llmd-vllm", "sglang", "vllm"), |
There was a problem hiding this comment.
🔴 The CI Lint job will fail on this PR: the new frameworks= line is 103 characters, over the 100-character format limit. ruff format --check rejects it because line-length = 100 in infx/ruff.toml. E501 is ignored, so ruff check passes, but the formatter check does not. Fix: run ruff format inferencex-e2e/infx so the formatter wraps the any_of(...) arguments. One argument per line with a trailing comma, matching the eval_unsets tuple at lines 80-83, is the result.
Why this was flagged
The changed line 54 in inferencex-e2e/infx/launch/drivers/srt/lanes.py is 8 spaces of indent plus frameworks=any_of("dynamo-sglang", "dynamo-trt", "dynamo-vllm", "llmd-vllm", "sglang", "vllm"),, which comes to 103 columns. inferencex-e2e/infx/ruff.toml:3 sets line-length = 100. .github/workflows/ci.yml:60 runs ruff@ latest format --check --diff inferencex-e2e/infx. The formatter would split this call across lines, so the check prints a diff and exits non-zero. On the base branch the line was 89 columns and the check passed. The E501 entry in the ruff.toml ignore list only silences the linter rule. It does not apply to ruff format --check, so it does not stop the failure. AGENTS.md also requires Ruff-formatted Python under infx/.
Verification: Line 54 of inferencex-e2e/infx/launch/drivers/srt/lanes.py is now 103 columns. inferencex-e2e/infx/ruff.toml:3 sets line-length = 100. E501 is in the lint ignore list (ruff.toml:10), so ruff check passes. But .github/workflows/ci.yml:60 runs ruff@ latest format --check --diff inferencex-e2e/infx, and the Lint matrix entry fails. On the base, the line was 85 columns and passed.
The gb200-nv multi-node srt-slurm lane only accepted Dynamo and llm-d frameworks, so recipes that front workers with srt-slurm's
sglang-routerorvllm-routerfrontend (frameworksglang/vllm) were refused before setup. GB300 already accepts any framework.This adds
sglangandvllmto the gb200-nvSRT_MULTIallowlist so the Kimi K3.1 GB200 SGLang and vLLM PRs can run with the vanilla routers instead of Dynamo, whose K3.1 build (ai-dynamo/dynamo#18124) does not compile.