Skip to content
This repository was archived by the owner on Mar 21, 2026. It is now read-only.

fix(num_devices): fix num_shard/num device auto compute when NVIDIA_VISIBLE_DEVICES == "all" or "void" - #3346

Merged
oOraph merged 2 commits into
mainfrom
fix_num_shard_auto_compute
Dec 18, 2025
Merged

oOraph merged 2 commits into
mainfrom
fix_num_shard_auto_compute

Conversation

@oOraph

@oOraph oOraph commented Dec 18, 2025 •

Copy link
Copy Markdown
Collaborator

If NVIDIA_VISIBLE_DEVICES == "all" then auto computed num shards was always 1
NVIDIA_VISIBLE_DEVICES == "void" is a new value coming from nvidia container runtime > 1.18

…ISIBLE_DEVICES == "all"

the computed num_shards was always 1 in this case, no matter what

Signed-off-by: Raphael Glon <oOraph@users.noreply.github.com>
@oOraph
oOraph requested review from danieldk and drbh December 18, 2025 15:06
drbh
drbh previously approved these changes Dec 18, 2025

@drbh drbh left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

…di mode

Signed-off-by: Raphael Glon <oOraph@users.noreply.github.com>
@oOraph oOraph changed the title fix(num_devices): fix num_shard/num device auto compute when NVIDIA_VISIBLE_DEVICES == "all" fix(num_devices): fix num_shard/num device auto compute when NVIDIA_VISIBLE_DEVICES == "all" or "void" Dec 18, 2025
@oOraph
oOraph merged commit dfb3fbe into main Dec 18, 2025
20 of 27 checks passed
@oOraph
oOraph deleted the fix_num_shard_auto_compute branch December 18, 2025 15:58
@julien-c

Copy link
Copy Markdown
Member

nice fix

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants