Skip to content

Commit 11773d7

Browse files
committed
fix(nemotron-vl): align recipe environments and verification docs
Signed-off-by: Chen Cui <chcui@nvidia.com>
1 parent a4b4757 commit 11773d7

11 files changed

Lines changed: 1873 additions & 9 deletions

File tree

‎docs/_static/model-verification/catalog-v1.json‎

Lines changed: 523 additions & 0 deletions
Large diffs are not rendered by default.

‎docs/fern/versions/nightly.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -130,6 +130,8 @@ navigation:
130130
path: ./nightly/pages/models/nemotron/nemotron3-super.mdx
131131
- page: Nemotron 3.5 Lightning
132132
path: ./nightly/pages/models/nemotron/nemotron3.5-lightning.mdx
133+
- page: Nemotron 3.5 Super VL
134+
path: ./nightly/pages/models/nemotron/nemotron3.5-super-vl.mdx
133135
- page: Nemotron 3 Ultra
134136
path: ./nightly/pages/models/nemotron/nemotron3-ultra.mdx
135137
- page: Nemotron Nano V2 VL

‎docs/fern/versions/nightly/pages/models/README.mdx‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44

55
Choose a model to open its interactive import/export and training configurations. Every option comes directly from an authoritative YAML verification card; independent fields are never combined into synthetic commands.
66

7-
Current inventory: **25 model cards** and **356 concrete configurations**.
7+
Current inventory: **26 model cards** and **374 concrete configurations**.
88

99
Each model page lets you select import/export, pretraining, benchmarking, SFT, LoRA, or long-context SFT and immediately see the exact command and expected result for the recorded precision and GPU.
1010

@@ -100,7 +100,7 @@ Choose a model to inspect its recorded import/export, training, and precision co
100100
<section class="verification-provider-card">
101101
<header class="verification-provider-heading">
102102
<strong>NVIDIA</strong>
103-
<span>6 models</span>
103+
<span>7 models</span>
104104
</header>
105105
<div class="verification-provider-models">
106106
<a class="verification-model-link" href="nemotron/nemotron-3-omni#verified-nemotron-3-nano-omni-30b-a3b-reasoning">
@@ -127,6 +127,10 @@ Choose a model to inspect its recorded import/export, training, and precision co
127127
<strong title="nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16">NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16</strong>
128128
<span class="verification-status verification-status--verified" title="Verified">✓ Verified</span>
129129
</a>
130+
<a class="verification-model-link" href="nemotron/nemotron3.5-super-vl#verified-nemotron-3.5-super-vl-120b-a12b">
131+
<strong title="nvidia/NVIDIA-Nemotron-3.5-Super-120B-A12B-SourceOfTruth">NVIDIA-Nemotron-3.5-Super-120B-A12B-SourceOfTruth</strong>
132+
<span class="verification-status verification-status--partial" title="Partial">◐ Partial</span>
133+
</a>
130134
</div>
131135
</section>
132136
<section class="verification-provider-card">

‎docs/fern/versions/nightly/pages/models/nemotron/nemotron3.5-super-vl.mdx‎

Lines changed: 627 additions & 0 deletions
Large diffs are not rendered by default.

‎docs/models/README.md‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44

55
Choose a model to open its interactive import/export and training configurations. Every option comes directly from an authoritative YAML verification card; independent fields are never combined into synthetic commands.
66

7-
Current inventory: **25 model cards** and **356 concrete configurations**.
7+
Current inventory: **26 model cards** and **374 concrete configurations**.
88

99
Each model page lets you select import/export, pretraining, benchmarking, SFT, LoRA, or long-context SFT and immediately see the exact command and expected result for the recorded precision and GPU.
1010

@@ -100,7 +100,7 @@ Choose a model to inspect its recorded import/export, training, and precision co
100100
<section class="verification-provider-card">
101101
<header class="verification-provider-heading">
102102
<strong>NVIDIA</strong>
103-
<span>6 models</span>
103+
<span>7 models</span>
104104
</header>
105105
<div class="verification-provider-models">
106106
<a class="verification-model-link" href="nemotron/nemotron-3-omni.html#verified-nemotron-3-nano-omni-30b-a3b-reasoning">
@@ -127,6 +127,10 @@ Choose a model to inspect its recorded import/export, training, and precision co
127127
<strong title="nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16">NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16</strong>
128128
<span class="verification-status verification-status--verified" title="Verified">✓ Verified</span>
129129
</a>
130+
<a class="verification-model-link" href="nemotron/nemotron3.5-super-vl.html#verified-nemotron-3.5-super-vl-120b-a12b">
131+
<strong title="nvidia/NVIDIA-Nemotron-3.5-Super-120B-A12B-SourceOfTruth">NVIDIA-Nemotron-3.5-Super-120B-A12B-SourceOfTruth</strong>
132+
<span class="verification-status verification-status--partial" title="Partial">◐ Partial</span>
133+
</a>
130134
</div>
131135
</section>
132136
<section class="verification-provider-card">

‎docs/models/nemotron/nemotron3.5-super-vl.md‎

Lines changed: 627 additions & 0 deletions
Large diffs are not rendered by default.

‎scripts/docs/generate_model_verification_catalog.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -106,6 +106,7 @@
106106
"nemotron-3-super-120b-a12b": "models/nemotron/nemotron3-super.md",
107107
"nemotron-3-ultra-550b-a55b": "models/nemotron/nemotron3-ultra.md",
108108
"nemotron-3.5-lightning": "models/nemotron/nemotron3.5-lightning.md",
109+
"nemotron-3.5-super-vl-120b-a12b": "models/nemotron/nemotron3.5-super-vl.md",
109110
"qwen3-235b-a22b": "models/qwen/qwen3-235b-a22b.md",
110111
"qwen3-30b-a3b": "models/qwen/qwen3-30b-a3b.md",
111112
"qwen3-8b": "models/qwen/qwen3-8b.md",

‎src/megatron/bridge/recipes/nemotron_omni/h100/nemotron_35_super_vl.py‎

Lines changed: 20 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -184,12 +184,20 @@ def nemotron_35_super_vl_pretrain_64gpu_h100_bf16_config() -> ConfigContainer:
184184
cfg.model.use_te_rng_tracker = False
185185
cfg.rng.te_rng_tracker = False
186186

187-
cfg.env_vars["NVTE_BWD_LAYERNORM_SM_MARGIN"] = 0
188-
cfg.env_vars["NVTE_FWD_LAYERNORM_SM_MARGIN"] = 0
189187
cfg.checkpoint.async_save = False
190188
cfg.model.freeze_language_model = False
191189
cfg.model.freeze_vision_model = False
192190
cfg.model.freeze_vision_projection = False
191+
cfg.env_vars = {
192+
**COMMON_RECIPE_ENV_VARS,
193+
"CUDA_DEVICE_MAX_CONNECTIONS": 32,
194+
"NUM_OF_HYBRID_EP_RANKS_PER_NVLINK_DOMAIN": 8,
195+
"NUM_OF_TOKENS_PER_CHUNK_COMBINE_API": 64,
196+
"NVLINK_DOMAIN_SIZE": 8,
197+
"USE_MNNVL": 0,
198+
"NVTE_BWD_LAYERNORM_SM_MARGIN": 0,
199+
"NVTE_FWD_LAYERNORM_SM_MARGIN": 0,
200+
}
193201
return cfg
194202

195203

@@ -231,6 +239,16 @@ def nemotron_35_super_vl_sft_64gpu_h100_bf16_config() -> ConfigContainer:
231239
cfg.optimizer.exp_avg_sq_dtype = torch.float32
232240
cfg.mixed_precision.grad_reduce_in_fp32 = True
233241
cfg.ddp.grad_reduce_in_fp32 = True
242+
cfg.env_vars = {
243+
**COMMON_RECIPE_ENV_VARS,
244+
"CUDA_DEVICE_MAX_CONNECTIONS": 32,
245+
"NUM_OF_HYBRID_EP_RANKS_PER_NVLINK_DOMAIN": 8,
246+
"NUM_OF_TOKENS_PER_CHUNK_COMBINE_API": 64,
247+
"NVLINK_DOMAIN_SIZE": 8,
248+
"USE_MNNVL": 0,
249+
"NVTE_BWD_LAYERNORM_SM_MARGIN": 20,
250+
"NVTE_FWD_LAYERNORM_SM_MARGIN": 20,
251+
}
234252
return cfg
235253

236254

@@ -347,8 +365,6 @@ def nemotron_35_super_vl_peft_16gpu_h100_bf16_config(
347365
cfg.env_vars = {
348366
**COMMON_RECIPE_ENV_VARS,
349367
"CUDA_DEVICE_MAX_CONNECTIONS": 1,
350-
"NVLINK_DOMAIN_SIZE": 8,
351-
"USE_MNNVL": 0,
352368
}
353369
return cfg
354370

‎src/megatron/bridge/recipes/nemotronh/h100/nemotron_3_super.py‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -265,7 +265,18 @@ def _nemotron_3_super_pretrain_64gpu_h100_bf16_config() -> ConfigContainer:
265265
BF16 pretraining configuration for 64 H100 GPUs.
266266
"""
267267
cfg = nemotron_3_super_pretrain_16gpu_h100_bf16_config()
268-
return _apply_nemotron_3_super_64gpu_h100_training_stack(cfg)
268+
cfg = _apply_nemotron_3_super_64gpu_h100_training_stack(cfg)
269+
cfg.env_vars = {
270+
**COMMON_RECIPE_ENV_VARS,
271+
"CUDA_DEVICE_MAX_CONNECTIONS": 32,
272+
"NUM_OF_HYBRID_EP_RANKS_PER_NVLINK_DOMAIN": 8,
273+
"NUM_OF_TOKENS_PER_CHUNK_COMBINE_API": 64,
274+
"NVLINK_DOMAIN_SIZE": 8,
275+
"USE_MNNVL": 0,
276+
"NVTE_BWD_LAYERNORM_SM_MARGIN": 20,
277+
"NVTE_FWD_LAYERNORM_SM_MARGIN": 20,
278+
}
279+
return cfg
269280

270281

271282
# =============================================================================

‎tests/unit_tests/doc_consistency/test_model_verification_catalog.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -271,6 +271,7 @@ def test_models_map_to_canonical_guides(generator: ModuleType, catalog: dict[str
271271
assert paths["muse-glimmer-30b"] == "models/muse_glimmer/muse-glimmer.md"
272272
assert paths["nemotron-3-nano-4b"] == "models/nemotron/nemotron3-nano-4b.md"
273273
assert paths["nemotron-3-super-120b-a12b"] == "models/nemotron/nemotron3-super.md"
274+
assert paths["nemotron-3.5-super-vl-120b-a12b"] == "models/nemotron/nemotron3.5-super-vl.md"
274275
assert paths["nemotron-3-ultra-550b-a55b"] == "models/nemotron/nemotron3-ultra.md"
275276
assert paths["qwen3.8-27b"] == "models/qwen/qwen3.8-27b.md"
276277

0 commit comments

Comments
 (0)