Skip to content

fix(ckpt): preserve MoE fine-tuning state and refresh Super VL verification #6220

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cuichenx merged 20 commits into from
Sep 28, 2026
Merged

fix(ckpt): preserve MoE fine-tuning state and refresh Super VL verification #6220

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
3a5b4f6
fix(nemotron): preserve shared-expert and MTP config semantics
cuichenx Sep 23, 2026
2866198
fix(nemotron): use the release-facing Super VL model identifier
cuichenx Sep 23, 2026
2449c2f
fix(nemotron-omni): avoid duplicate media prompt newlines
cuichenx Sep 23, 2026
00e022f
fix(nemotron-omni): preserve reasoning parsers during export
cuichenx Sep 23, 2026
70cc4d3
fix(inference): reuse an active prebuilt launcher environment
cuichenx Sep 23, 2026
5afd440
fix(training): use native expert-bias padding masking
cuichenx Sep 23, 2026
fb082ba
fix(peft): preserve learned router expert-bias buffers
cuichenx Sep 23, 2026
e2cf6cb
fix(inference): disable training token dropping on checkpoint load
cuichenx Sep 23, 2026
7f4f96c
fix(inference): disable NCCL-EP zero-copy for dropless loading
cuichenx Sep 25, 2026
2cc5028
test(peft): cover and document router-bias checkpoint compatibility
cuichenx Sep 25, 2026
7ef2d38
fix(inference): use dropless routing for saved VLM checkpoints
cuichenx Sep 25, 2026
616de84
fix(nemotron): use final Super VL checkpoint name
cuichenx Sep 25, 2026
a51a215
fix(inference): disable synthetic expert routing bias
cuichenx Sep 25, 2026
5e63b9d
chore: merge main for verification refresh
cuichenx Sep 25, 2026
5639b78
fix(inference): disable paged stashing for dropless loads
cuichenx Sep 25, 2026
9e6499e
fix(verification): recognize literal chat tokens and loopback addresses
cuichenx Sep 27, 2026
d58132e
docs(verification): refresh Super VL and text-only evidence
cuichenx Sep 28, 2026
dcc4ee7
fix(nemotron): preserve physical HF MTP block count
cuichenx Sep 28, 2026
52d80a3
docs(checkpoint): explain nullable config overrides
cuichenx Sep 28, 2026
4d7ab0b
fix(test): align text-only MTP expectations and display names
cuichenx Sep 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
834 changes: 680 additions & 154 deletions docs/_static/model-verification/catalog-v1.json
View file
Open in desktop

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions docs/fern/versions/nightly.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,8 @@ navigation:
path: ./nightly/pages/models/nemotron/nemotron3.5-lightning.mdx
- page: Nemotron 3.5 Super VL
path: ./nightly/pages/models/nemotron/nemotron3.5-super-vl.mdx
- page: Nemotron-3.5 Super Text-only
path: ./nightly/pages/models/nemotron/nemotron3.5-super-vl-text-only.mdx
- page: Nemotron 3 Ultra
path: ./nightly/pages/models/nemotron/nemotron3-ultra.mdx
- page: Nemotron Nano V2 VL
Expand Down
10 changes: 7 additions & 3 deletions docs/fern/versions/nightly/pages/models/README.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

Choose a model to open its interactive import/export and training configurations. Every option comes directly from an authoritative YAML verification card; independent fields are never combined into synthetic commands.

Current inventory: **27 model cards** and **387 concrete configurations**.
Current inventory: **28 model cards** and **405 concrete configurations**.

Each model page lets you select import/export, pretraining, benchmarking, SFT, LoRA, or long-context SFT and immediately see the exact command and expected result for the recorded precision and GPU.

Expand Down Expand Up @@ -112,7 +112,7 @@ Choose a model to inspect its recorded import/export, training, and precision co
<section class="verification-provider-card">
<header class="verification-provider-heading">
<strong>NVIDIA</strong>
<span>7 models</span>
<span>8 models</span>
</header>
<div class="verification-provider-models">
<a class="verification-model-link" href="nemotron/nemotron-3-omni#verified-nemotron-3-nano-omni-30b-a3b-reasoning">
Expand Down Expand Up @@ -140,7 +140,11 @@ Choose a model to inspect its recorded import/export, training, and precision co
<span class="verification-status verification-status--verified" title="Verified">✓ Verified</span>
</a>
<a class="verification-model-link" href="nemotron/nemotron3.5-super-vl#verified-nemotron-3.5-super-vl-120b-a12b">
<strong title="nvidia/NVIDIA-Nemotron-3.5-Super-120B-A12B">NVIDIA-Nemotron-3.5-Super-120B-A12B</strong>
<strong title="nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16">NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16</strong>
<span class="verification-status verification-status--partial" title="Partial">◐ Partial</span>
</a>
<a class="verification-model-link" href="nemotron/nemotron3.5-super-vl-text-only#verified-nemotron-3.5-super-vl-120b-a12b-text-only">
<strong title="nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16">NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16</strong>
<span class="verification-status verification-status--partial" title="Partial">◐ Partial</span>
</a>
</div>
Expand Down
2 changes: 2 additions & 0 deletions docs/fern/versions/nightly/pages/models/nemotron/index.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ Nemotron model documentation is organized by model variant.
| Nemotron-3 Nano 4B | [nemotron3-nano-4b.md](nemotron3-nano-4b.md) |
| Nemotron-3 Super | [nemotron3-super.md](nemotron3-super.md) |
| Nemotron-3.5 Lightning | [nemotron3.5-lightning.md](nemotron3.5-lightning.md) |
| Nemotron-3.5 Super VL | [nemotron3.5-super-vl.md](nemotron3.5-super-vl.md) |
| Nemotron-3.5 Super Text-only | [nemotron3.5-super-vl-text-only.md](nemotron3.5-super-vl-text-only.md) |
| Nemotron-3 Ultra | [nemotron3-ultra.md](nemotron3-ultra.md) |
| Nemotron Nano V2 VL | [nemotron-nano-v2-vl.md](nemotron-nano-v2-vl.md) |
| Nemotron-3 Nano Omni | [nemotron-3-omni.md](nemotron-3-omni.md) |
Expand Down
613 changes: 613 additions & 0 deletions .../fern/versions/nightly/pages/models/nemotron/nemotron3.5-super-vl-text-only.mdx
View file
Open in desktop

Large diffs are not rendered by default.

136 changes: 66 additions & 70 deletions docs/fern/versions/nightly/pages/models/nemotron/nemotron3.5-super-vl.mdx
View file
Open in desktop

Large diffs are not rendered by default.

10 changes: 7 additions & 3 deletions docs/models/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

Choose a model to open its interactive import/export and training configurations. Every option comes directly from an authoritative YAML verification card; independent fields are never combined into synthetic commands.

Current inventory: **27 model cards** and **387 concrete configurations**.
Current inventory: **28 model cards** and **405 concrete configurations**.

Each model page lets you select import/export, pretraining, benchmarking, SFT, LoRA, or long-context SFT and immediately see the exact command and expected result for the recorded precision and GPU.

Expand Down Expand Up @@ -112,7 +112,7 @@ Choose a model to inspect its recorded import/export, training, and precision co
<section class="verification-provider-card">
<header class="verification-provider-heading">
<strong>NVIDIA</strong>
<span>7 models</span>
<span>8 models</span>
</header>
<div class="verification-provider-models">
<a class="verification-model-link" href="nemotron/nemotron-3-omni.html#verified-nemotron-3-nano-omni-30b-a3b-reasoning">
Expand Down Expand Up @@ -140,7 +140,11 @@ Choose a model to inspect its recorded import/export, training, and precision co
<span class="verification-status verification-status--verified" title="Verified">✓ Verified</span>
</a>
<a class="verification-model-link" href="nemotron/nemotron3.5-super-vl.html#verified-nemotron-3.5-super-vl-120b-a12b">
<strong title="nvidia/NVIDIA-Nemotron-3.5-Super-120B-A12B">NVIDIA-Nemotron-3.5-Super-120B-A12B</strong>
<strong title="nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16">NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16</strong>
<span class="verification-status verification-status--partial" title="Partial">◐ Partial</span>
</a>
<a class="verification-model-link" href="nemotron/nemotron3.5-super-vl-text-only.html#verified-nemotron-3.5-super-vl-120b-a12b-text-only">
<strong title="nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16">NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16</strong>
<span class="verification-status verification-status--partial" title="Partial">◐ Partial</span>
</a>
</div>
Expand Down
2 changes: 2 additions & 0 deletions docs/models/nemotron/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ nemotron3-nano-4b.md
nemotron3-super.md
nemotron3.5-lightning.md
nemotron3.5-super-vl.md
nemotron3.5-super-vl-text-only.md
nemotron3-ultra.md
nemotron-nano-v2-vl.md
nemotron-3-omni.md
Expand All @@ -26,6 +27,7 @@ nemotron-3-omni.md
| Nemotron-3 Super | [nemotron3-super.md](nemotron3-super.md) |
| Nemotron-3.5 Lightning | [nemotron3.5-lightning.md](nemotron3.5-lightning.md) |
| Nemotron-3.5 Super VL | [nemotron3.5-super-vl.md](nemotron3.5-super-vl.md) |
| Nemotron-3.5 Super Text-only | [nemotron3.5-super-vl-text-only.md](nemotron3.5-super-vl-text-only.md) |
| Nemotron-3 Ultra | [nemotron3-ultra.md](nemotron3-ultra.md) |
| Nemotron Nano V2 VL (deprecated) | [nemotron-nano-v2-vl.md](nemotron-nano-v2-vl.md) |
| Nemotron-3 Nano Omni | [nemotron-3-omni.md](nemotron-3-omni.md) |
Expand Down
Loading
Loading