Skip to content

Commit 6b7602e

Browse files
committed
docs(verification): record Qwen 3.8 GB200 throughput
Signed-off-by: Chen Cui <chcui@nvidia.com>
1 parent bdab753 commit 6b7602e

1 file changed

Lines changed: 46 additions & 6 deletions

File tree

  • examples/model_verification_cards/qwen3.8-27b

‎examples/model_verification_cards/qwen3.8-27b/card.yaml‎

Lines changed: 46 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,12 @@
33

44
title: qwen3_8_27b
55
summary: >
6-
Performance disclaimer: this model has not been performance-tuned; reported
7-
timing and throughput metrics are sanity checks, not optimized performance
8-
results. All workloads in this card target GB200 hardware. Conversion and
9-
deterministic inference and all required functional training gates are
10-
verified. No dense-27B GB200 performance or Megatron FSDP recipe is
11-
available.
6+
Performance scope: pretrain_performance.GB200 uses the tuned canonical
7+
projector-only MockVLM recipe and records 451.720 model TFLOP/s/GPU on 16
8+
GB200 GPUs; timing from all functional items remains sanity evidence. All
9+
workloads in this card target GB200 hardware. Conversion, deterministic
10+
inference, and all required functional training gates are verified. No
11+
dense-27B Megatron FSDP recipe is available.
1212
verification_index:
1313
model_level:
1414
verified:
@@ -21,6 +21,8 @@ verification_index:
2121
training:
2222
GB200:
2323
verified: [pretrain, sft, sft_export_inference, sft_long_context, peft, checkpoint_resume]
24+
performance:
25+
GB200: verified
2426
model:
2527
hf_id: Qwen/Qwen3.8-27B
2628
hf_revision: 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0 # pragma: allowlist secret
@@ -428,3 +430,41 @@ items:
428430
totaling 56,110,835,024 bytes, its tracker records iteration 20, both
429431
built-in evaluation passes completed at loss 5.043188, and the process
430432
exited successfully in 17m27s.
433+
434+
pretrain_performance:
435+
GB200:
436+
status: verified
437+
precision: bf16
438+
bridge_commit: bdab7530dde9196c24d580bd3bb6ee23c0e8323a # pragma: allowlist secret
439+
command: >
440+
./scripts/training/train.sh --nodes 4 --gpus-per-node 4
441+
--additional-slurm-params segment=4
442+
--recipe qwen35_vl_27b_pretrain_16gpu_gb200_bf16_mock_config
443+
--mode pretrain --pretrained_checkpoint
444+
work/model-verification/qwen3.8-27b/gpu-megatron/iter_0000000
445+
--max_steps 20 --warmup_iters 2
446+
dataset.hf_processor_path=work/model-verification/qwen3.8-27b/gpu-hf-export
447+
tokenizer.tokenizer_model=work/model-verification/qwen3.8-27b/gpu-hf-export
448+
model.hf_model_id=Qwen/Qwen3.8-27B model.bos_token_id=248044
449+
logger.save_config_filepath=work/model-verification/qwen3.8-27b/pretrain-performance/resolved-config.yaml
450+
scheduler.lr_decay_iters=20
451+
last_verified: 2026-08-14
452+
metrics:
453+
initial_loss: 6.853174
454+
final_loss: 4.975722
455+
last_10_steps_step_time_ms_avg: 3105.340
456+
last_10_steps_model_tflops_per_gpu_avg: 451.720
457+
peak_allocated_memory_gib: 42.244
458+
expected_result: >
459+
On 16 GB200 GPUs in one NVL72 domain, the exact canonical BF16
460+
projector-only MockVLM recipe completes 20 steps at TP4/PP2/CP2,
461+
sequence length 4096, and MBS4/GBS32. Language and vision weights are
462+
frozen while the multimodal projector trains. The recipe keeps fused
463+
Gated DeltaNet execution enabled, disables unnecessary activation
464+
recompute, evaluation, and checkpoint output, and retains NaN and
465+
large-gradient checks. Loss remains finite from 6.853174 to 4.975722
466+
with zero skipped or NaN iterations. Excluding first-iteration kernel
467+
compilation, steps 11-20 average 3,105.340 ms and 451.720 model
468+
TFLOP/s/GPU; peak allocated memory is 42.244 GiB. The process exits
469+
successfully. This mock projector-only result is throughput evidence,
470+
not full-model convergence evidence.

0 commit comments

Comments
 (0)