Repository navigation
perf(recipe): add an 8-GPU GB300 MXFP8 pretrain recipe for Qwen3.5-VL 35B-A3B - #6413
Merged
Merged
Conversation
… 35B-A3B Add qwen35_vl_35b_a3b_pretrain_8gpu_gb300_fp8mx_library_config, an MXFP8 Qwen3.5-VL 35B-A3B pretraining recipe for 8 GB300 GPUs. It starts from qwen35_vl_35b_a3b_pretrain_16gpu_gb300_bf16_config, keeps its parallel layout, CUDA graphs, recompute, batch shape and real routing, and adds: - MXFP8 with FP8 parameter all-gather (bf16_with_mxfp8_mixed); - Transformer Engine's CuTe DSL fused grouped MLP: the op fuser, the 32-wide GLU interleave and the NVTE_CUTEDSL_FUSED_GROUPED_MLP gate, which only work together; - 32 flex-dispatcher SMs, with HybridEP's fused permute off: it requires identical dispatch, combine and preprocessing chunk sizes, and this recipe uses 64/128/64; - the HybridEP preprocessing SMs, dispatch/expert-wgrad overlap, dispatch and preprocessing chunk sizes and cuDNN normalization of the configuration it was tuned on, which the 16-GPU recipe does not carry over; - cuDNN GDN (gdn_kernel_backend="transformer_engine"), set only when Megatron-Core declares the field (NVIDIA/Megatron-LM#7583), with the same guard as _enable_gdn_conv_fusion. The name carries a _library suffix because qwen35_vl_35b_a3b_pretrain_8gpu_gb300_fp8mx_config is already a perf recipe. Transformer Engine reads NVTE_CUTEDSL_FUSED_GROUPED_MLP once, when it registers the fused MLP at import time, and run_recipe.py applied a library recipe's env_vars after that import, so the gate was ignored. Add the variable to IMPORT_TIME_RECIPE_ENV_VARS, so that a recipe declaring it goes through the existing re-exec bootstrap, as for NVTE_CPU_OFFLOAD_V1. Through run_recipe.py this also changes two existing GB200 library recipes that declare the gate and no other import-time variable: qwen35_text_35b_a3b_sft_long_context_16gpu_gb200_fp8mx_config, which enables the op fuser and the GLU interleave, now runs the fused MLP instead of the unfused grouped GEMM (not measured here), and qwen3_30b_a3b_pretrain_8gpu_gb200_fp8mx_config, which keeps the op fuser off, now re-executes once. Tests check the recipe's settings, the exact set of model fields and environment variables in which it differs from the 16-GPU recipe, that building it leaves the 16-GPU recipe unchanged, that cuDNN GDN is selected for a model config that declares the field and that no attribute is created on one without it, and that the bootstrap covers both import-time variables. Signed-off-by: tomlifu <lifuz@nvidia.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cuichenx
added
feature
malay-nagda
reviewed
Oct 12, 2026
malay-nagda
requested changes
Oct 12, 2026
cuichenx
added
waiting-on-customer
Address review: qwen35_vl_35b_a3b_pretrain_8gpu_gb300_fp8mx_library_config no longer edits the env_vars it inherits from qwen35_vl_35b_a3b_pretrain_16gpu_gb300_bf16_config. It declares its whole environment in one cfg.env_vars literal, COMMON_RECIPE_ENV_VARS first, right before it returns, as hardware recipes do, so the recipe shows every variable it sets. The environment is unchanged: the five variables the recipe used to overwrite or add (NVTE_CUTEDSL_FUSED_GROUPED_MLP, the dispatch and preprocessing chunk sizes and the two cuDNN normalization flags) keep their values, and every other variable keeps the 16-GPU value. test_representative_recipe_environment_is_visible now covers this recipe, so a recipe that edits inherited env_vars again fails there. Signed-off-by: tomlifu <lifuz@nvidia.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
/ok to test f062cb9 |
Contributor
|
Automatic Claude reviews have been retired. To request a pull-request review, post a comment containing: Add |
cuichenx
added
the
r0.7.0
cuichenx
approved these changes
Oct 12, 2026
malay-nagda
approved these changes
Oct 13, 2026
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add qwen35_vl_35b_a3b_pretrain_8gpu_gb300_fp8mx_library_config, an MXFP8 Qwen3.5-VL 35B-A3B pretraining recipe for 8 GB300 GPUs. It starts from qwen35_vl_35b_a3b_pretrain_16gpu_gb300_bf16_config, keeps its parallel layout, CUDA graphs, recompute, batch shape and real routing, and adds:
Changelog
GitHub Actions CI
See the CI section in the Contributing doc for how to trigger the CI. A Nvidia developer will need to approve and trigger the CI for external contributors.
Before your PR is "Ready for review"
Pre checks:
If you haven't finished some of the above items you can still open "Draft" PR.
Additional Information