Add p6-b300 and p6e-gb200 GPU health-check instance profiles - #1201
Open
Zhenye-Na wants to merge 1 commit into
Open
Add p6-b300 and p6e-gb200 GPU health-check instance profiles#1201Zhenye-Na wants to merge 1 commit into
Zhenye-Na wants to merge 1 commit into
Conversation
The GPU cluster health check validates detected hardware against a per-instance profile in instance-profiles.conf. When an instance type has no matching row, the loader falls back to a zero GPU/EFA count and the GPU-count, EFA-enumeration, and topology checks silently skip, so a run on an unrecognized node reports PASS while validating almost nothing. Add profiles for the Blackwell-generation instances: - p6-b300.48xlarge: 8 GPU, 16 EFA, NVLink (17 network cards, NCI 0 ENA-only and NCI 1-16 EFA-only). - p6e-gb200.36xlarge: 4 GPU, 4 EFA, NVLink (2 GB200 superchips; recommended max-EFA layout of 4 x 400 Gbps interfaces). Also add AWS documentation links for the GPU and EFA/network-card counts to the file header. Values verified against: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/efa-acc-inst-types.html Signed-off-by: Nathan Na <nzhenye@amazon.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
The GPU health check compares each node's real hardware against a per-instance profile in
instance-profiles.conf. If an instance type isn't in that file, the loader falls back to a GPU/EFA count of 0, and every check guarded by> 0(GPU count, EFA enumeration, topology) just skips. So the suite happily reports PASS on an unrecognized node while barely checking anything.The file currently stops at
p6-b200, which means the newer Blackwell nodes fall into exactly that gap.Changes
Added profiles for two Blackwell-generation instances:
p6-b300.48xlarge→8|16|true|efa— 8 GPUs, NVLink. The AWS EFA docs list 17 network cards: NCI 0 is ENA-only and NCI 1–16 are EFA-only, so 16 EFA devices.p6e-gb200.36xlarge→4|4|true|efa— 4 GPUs (2× GB200 superchips), NVLink (NVL72 domain). GB200's attached EFA count is provisioning-dependent;4is the AWS-recommended max-EFA layout (4 × 400 Gbps). Called out in an inline comment.I also added the AWS doc links to the file header so the counts are traceable.
This is a data-only change — no code touched, existing rows untouched, so nothing changes for already-supported instances.
Source: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/efa-acc-inst-types.html
Test Plan
It's a config change, so I checked that the rows are well-formed and that the loader can actually find them.
Test Results
Both new types resolve to a non-zero profile, so the GPU/EFA/topology checks now run on these nodes instead of skipping:
I don't have p6-b300 / p6e-gb200 hardware on hand, so a live run of the suite on real nodes would be a good follow-up for anyone with fleet access.
Directory Structure
N/A — updates an existing config file, not a test case.
Checklist
mainbranch.latest). — N/A, no dependencies