feat(recipes): modernize DeepSeek V4 library configs

[cuichenx]

What does this PR do?

• modernizes the DeepSeek V4 GB200 library recipes while leaving perf_recipes unchanged
• adds a 128-GPU GB200 MXFP8 pretraining recipe with PP1/EP64, grouped GEMM, HybridEP, selective recompute, attention activation offload, and MXFP8 parameter gather/buffer reuse
• adds a GB200 offline-packed OpenMath SFT recipe with grouped GEMM, HybridEP, fixed packed-sequence shapes, DSA execution fusion, selective recompute, and attention activation offload
• keeps natural routing, unlimited expert capacity, full-precision gradient reduction, validation checks, optimizer semantics, and checkpoint behavior; it does not port forced balancing, paged stash, token dropping, reduced-precision gradients, or CUDA graphs into the library recipe
• applies recipe environment variables before training imports for both library and benchmark recipes
• preserves legacy DeepSeek V4 indexer scorer names during BF16 export
• adds focused recipe, runner, conversion, and verification-card coverage
• adds the DeepSeek V4 Flash verification card with the final pretraining and SFT evidence

DeepSeek V4 pretraining requires Megatron-LM dev. Grouped-FP8 checkpoint saves also require the empty extra-state fix from NVIDIA/Megatron-LM#5997. Until that fix reaches dev, use the equivalent signed commit from NVIDIA/Megatron-LM#6509 on a compatible dev checkout.

Validation

Workload	Configuration	Result	Last-10-step average
Pretrain	128 GB200, MXFP8, PP1/EP64, natural routing	100 finite steps; grouped-MXFP8 checkpoint at step 100; fresh model/optimizer/RNG restore completed steps 101-105	131.36 TFLOP/s/GPU, 5713.47 ms
Packed SFT	32 GB200, BF16, PP4/EP8	100 finite steps; fresh checkpoint reload completed step 101	42.18 TFLOP/s/GPU, 8575.85 ms

Both runs completed with zero skipped or NaN iterations. The pretraining loss moved from 12.59201 to 5.960634; packed-SFT loss moved from 0.9435847 to 0.2834835.

Additional checks:

• focused branch test matrix: 298 passed, 1 skipped
• latest-main integrated focused matrix: 335 passed, 1 skipped, 1 deselected
• grouped checkpoint tests: 15 passed
• all-file pre-commit: passed
• verification-card validator: passed on the latest-main integrated branch
• git diff --check and public-diff privacy scan: passed

The deselected metadata test has an unrelated Nemotron 3 Super recipe-name collision that reproduces on current main; this PR does not modify that metadata or either Nemotron recipe export.

Verification scope

The card marks GB200 pretraining and packed SFT verified. Formal checkpoint-resume comparison remains unverified because its contract requires an independent step-50-to-100 run with step-51 and step-100 sentinels. Public post-SFT export/inference, long-context SFT, and the canonical mock-data performance item also remain explicitly unverified.

A capacity-1.5 performance screen was rejected for the library recipe because it dropped more than 64% of expert assignments and routing probability mass. The verified library recipe therefore retains unlimited-capacity natural routing.

• I, the PR author, have personally reviewed every line of this PR.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.