Skip to content

Commit 19a7d6d

Browse files
committed
docs(models): re-verify Nemotron Super H100 long-context SFT at a recipe-containing revision
The sft_long_context.H100 item was marked verified at 44f3031, a revision where its --recipe nemotron_3_super_sft_16gpu_h100_bf16_32k_config did not exist (only the earlier _lowmem_ name did). Drop the item pin so it inherits the card default ffeb5a7, which contains the recipe, and record the date and metrics from a fresh 16xH100 run of the published command at that revision. Regenerate the verification catalog and model pages. Detected by: megatron-bridge QA (NVBug 6631246) Signed-off-by: Pruthviraj Prakash <pruprakash@nvidia.com>
1 parent 7911a42 commit 19a7d6d

4 files changed

Lines changed: 25 additions & 26 deletions

File tree

‎docs/_static/model-verification/catalog-v1.json‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -4818,7 +4818,7 @@
48184818
"workflow": "sft_export_inference"
48194819
},
48204820
{
4821-
"bridge_commit": "44f3031a2e77ff455eb128efbaa19322ad6397a6",
4821+
"bridge_commit": "ffeb5a7ae817a5a6ccfa3850d7dee937faad7f0b",
48224822
"command_topologies": [
48234823
{
48244824
"gpus_per_node": "8",
@@ -4837,15 +4837,15 @@
48374837
"sequence_packing": "offline"
48384838
},
48394839
"entry_id": "nemotron-3-super-120b-a12b-sft-long-context-h100",
4840-
"expected_result": "On 16 H100s, the dedicated TP1/PP8/CP2/EP2 recipe completes exactly ten packed 32K SFT steps with pad-4 alignment, full recompute, and full optimizer CPU offload. Loss is finite from 1.168852 to 1.008992 with no skipped or NaN iterations. The ten-step metric window includes the first-step cold-start cost by policy, the resolved configuration is persisted, and the final model-only checkpoint saves successfully.\n",
4840+
"expected_result": "On 16 H100s, the dedicated TP1/PP8/CP2/EP2 recipe completes exactly ten packed 32K SFT steps with pad-4 alignment, full recompute, and full optimizer CPU offload. Loss is finite from 1.168852 to 1.011637 with no skipped or NaN iterations. The ten-step metric window includes the first-step cold-start cost by policy, the resolved configuration is persisted, and the final model-only checkpoint saves successfully.\n",
48414841
"hardware": "H100",
4842-
"last_verified": "2026-07-24",
4842+
"last_verified": "2026-10-08",
48434843
"metrics": {
4844-
"final_loss": 1.008992,
4844+
"final_loss": 1.011637,
48454845
"initial_loss": 1.168852,
4846-
"last_10_steps_model_tflops_per_gpu_avg": 5.58,
4847-
"last_10_steps_step_time_ms_avg": 132859.58,
4848-
"last_10_steps_tokens_per_second_per_gpu_avg": 30.83
4846+
"last_10_steps_model_tflops_per_gpu_avg": 6.89,
4847+
"last_10_steps_step_time_ms_avg": 123114.4,
4848+
"last_10_steps_tokens_per_second_per_gpu_avg": 33.27
48494849
},
48504850
"precision": "bf16",
48514851
"source_pointer": "items.sft_long_context.H100",

‎docs/fern/versions/nightly/pages/models/nemotron/nemotron3-super.mdx‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -462,7 +462,7 @@ Choose a workflow, precision, and exact recorded combination. The command and ex
462462
<dl class="verification-model-detail-meta">
463463
<div><dt>Hardware</dt><dd>H100</dd></div>
464464
<div><dt>Precision</dt><dd>BF16</dd></div>
465-
<div><dt>Last verified</dt><dd>2026-07-24</dd></div>
465+
<div><dt>Last verified</dt><dd>2026-10-08</dd></div>
466466
</dl>
467467
<section class="verification-recorded-metrics">
468468
<h5>Recorded metrics</h5>
@@ -473,19 +473,19 @@ Choose a workflow, precision, and exact recorded combination. The command and ex
473473
</div>
474474
<div>
475475
<dt>Final loss</dt>
476-
<dd>1.008992</dd>
476+
<dd>1.011637</dd>
477477
</div>
478478
<div>
479479
<dt>Step time · last 10 avg</dt>
480-
<dd>132,859.580 ms</dd>
480+
<dd>123,114.400 ms</dd>
481481
</div>
482482
<div>
483483
<dt>Model throughput · last 10 avg</dt>
484-
<dd>5.580 TFLOP/s/GPU</dd>
484+
<dd>6.890 TFLOP/s/GPU</dd>
485485
</div>
486486
<div>
487487
<dt>Token throughput · last 10 avg</dt>
488-
<dd>30.830 tokens/s/GPU</dd>
488+
<dd>33.270 tokens/s/GPU</dd>
489489
</div>
490490
</dl>
491491
</section>
@@ -501,7 +501,7 @@ Choose a workflow, precision, and exact recorded combination. The command and ex
501501
</section>
502502
<section class="verification-expected-result">
503503
<h5>Expected result</h5>
504-
<p>On 16 H100s, the dedicated TP1/PP8/CP2/EP2 recipe completes exactly ten packed 32K SFT steps with pad-4 alignment, full recompute, and full optimizer CPU offload. Loss is finite from 1.168852 to 1.008992 with no skipped or NaN iterations. The ten-step metric window includes the first-step cold-start cost by policy, the resolved configuration is persisted, and the final model-only checkpoint saves successfully.</p>
504+
<p>On 16 H100s, the dedicated TP1/PP8/CP2/EP2 recipe completes exactly ten packed 32K SFT steps with pad-4 alignment, full recompute, and full optimizer CPU offload. Loss is finite from 1.168852 to 1.011637 with no skipped or NaN iterations. The ten-step metric window includes the first-step cold-start cost by policy, the resolved configuration is persisted, and the final model-only checkpoint saves successfully.</p>
505505
</section>
506506
</article>
507507
<article id="nemotron-3-super-120b-a12b-sft-long-context-gb200" class="verification-model-detail" data-entry-detail="nemotron-3-super-120b-a12b-sft-long-context-gb200" tabindex="-1">

‎docs/models/nemotron/nemotron3-super.md‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -470,7 +470,7 @@ Choose a workflow, precision, and exact recorded combination. The command and ex
470470
<dl class="verification-model-detail-meta">
471471
<div><dt>Hardware</dt><dd>H100</dd></div>
472472
<div><dt>Precision</dt><dd>BF16</dd></div>
473-
<div><dt>Last verified</dt><dd>2026-07-24</dd></div>
473+
<div><dt>Last verified</dt><dd>2026-10-08</dd></div>
474474
</dl>
475475
<section class="verification-recorded-metrics">
476476
<h5>Recorded metrics</h5>
@@ -481,19 +481,19 @@ Choose a workflow, precision, and exact recorded combination. The command and ex
481481
</div>
482482
<div>
483483
<dt>Final loss</dt>
484-
<dd>1.008992</dd>
484+
<dd>1.011637</dd>
485485
</div>
486486
<div>
487487
<dt>Step time · last 10 avg</dt>
488-
<dd>132,859.580 ms</dd>
488+
<dd>123,114.400 ms</dd>
489489
</div>
490490
<div>
491491
<dt>Model throughput · last 10 avg</dt>
492-
<dd>5.580 TFLOP/s/GPU</dd>
492+
<dd>6.890 TFLOP/s/GPU</dd>
493493
</div>
494494
<div>
495495
<dt>Token throughput · last 10 avg</dt>
496-
<dd>30.830 tokens/s/GPU</dd>
496+
<dd>33.270 tokens/s/GPU</dd>
497497
</div>
498498
</dl>
499499
</section>
@@ -509,7 +509,7 @@ Choose a workflow, precision, and exact recorded combination. The command and ex
509509
</section>
510510
<section class="verification-expected-result">
511511
<h5>Expected result</h5>
512-
<p>On 16 H100s, the dedicated TP1/PP8/CP2/EP2 recipe completes exactly ten packed 32K SFT steps with pad-4 alignment, full recompute, and full optimizer CPU offload. Loss is finite from 1.168852 to 1.008992 with no skipped or NaN iterations. The ten-step metric window includes the first-step cold-start cost by policy, the resolved configuration is persisted, and the final model-only checkpoint saves successfully.
512+
<p>On 16 H100s, the dedicated TP1/PP8/CP2/EP2 recipe completes exactly ten packed 32K SFT steps with pad-4 alignment, full recompute, and full optimizer CPU offload. Loss is finite from 1.168852 to 1.011637 with no skipped or NaN iterations. The ten-step metric window includes the first-step cold-start cost by policy, the resolved configuration is persisted, and the final model-only checkpoint saves successfully.
513513
</p>
514514
</section>
515515
</article>

‎examples/model_verification_cards/nemotron-3-super-120b-a12b/card.yaml‎

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -399,7 +399,6 @@ items:
399399
H100:
400400
status: verified
401401
precision: bf16
402-
bridge_commit: 44f3031a2e77ff455eb128efbaa19322ad6397a6 # pragma: allowlist secret
403402
enabled_features:
404403
sequence_packing: offline
405404
context_parallel_size: 2
@@ -431,17 +430,17 @@ items:
431430
--save_dir work/model-verification/nemotron-3-super-120b-a12b/h100-long-sft
432431
--save_interval 10 logger.log_interval=1 logger.log_throughput=true
433432
logger.tensorboard_dir=null
434-
last_verified: 2026-07-24
433+
last_verified: 2026-10-08
435434
metrics:
436435
initial_loss: 1.168852
437-
final_loss: 1.008992
438-
last_10_steps_step_time_ms_avg: 132859.580
439-
last_10_steps_model_tflops_per_gpu_avg: 5.580
440-
last_10_steps_tokens_per_second_per_gpu_avg: 30.830
436+
final_loss: 1.011637
437+
last_10_steps_step_time_ms_avg: 123114.400
438+
last_10_steps_model_tflops_per_gpu_avg: 6.890
439+
last_10_steps_tokens_per_second_per_gpu_avg: 33.270
441440
expected_result: >
442441
On 16 H100s, the dedicated TP1/PP8/CP2/EP2 recipe completes exactly
443442
ten packed 32K SFT steps with pad-4 alignment, full recompute, and full
444-
optimizer CPU offload. Loss is finite from 1.168852 to 1.008992 with no
443+
optimizer CPU offload. Loss is finite from 1.168852 to 1.011637 with no
445444
skipped or NaN iterations. The ten-step metric window includes the
446445
first-step cold-start cost by policy, the resolved configuration is
447446
persisted, and the final model-only checkpoint saves successfully.

0 commit comments

Comments
 (0)