Repository navigation
fix(recipe): give Moonlight-16B 8-chunk pipeline layouts 27 decoder layers - #6359
Conversation
…ayers The PP=8/VP=1 and PP=4/VP=2 layouts in _get_moonlight_pipeline_layout described 4 + 4*6 + 3 = 31 decoder layers for a 27-layer model, so Megatron-Core's validate_layer_layout rejects them at config finalization. Re-split both as a front-loaded [4, 4, 4, 3, 3, 3, 3, 3]. Adds a unit test that runs every supported Moonlight layout through Megatron-Core layout validation at the model depth (red-green verified). Signed-off-by: Pruthviraj Prakash <pruprakash@nvidia.com>
pruprakash
added
bug
|
/review |
svcnvidia-nemo-ci
left a comment
svcnvidia-nemo-ci
left a comment
There was a problem hiding this comment.
No verified findings. Both updated layouts now contain eight chunks totaling 27 decoder layers, retaining embedding and loss at the endpoints. The new regression test covers all five supported explicit layouts and checks inferred VP size.
Static review only; tests were not executed.
Suggested test cases
- test_moonlight_pipeline_layouts_match_model_depth: all supported explicit layouts should validate at 27 layers, particularly PP=8/VP=1 and PP=4/VP=2.
- test_moonlight_pipeline_layout_tracks_supported_runner_override: retain layout rebuilding after public PP/VP overrides and the existing data-parallel-size and HybridEP-padding assertions.
- test_moonlight_16b_legacy_sft_contract_remains_available: retain the unchanged PP=2 split of 14/13 decoder layers.
Review generated by codex
|
/ok to test 862874f |
|
Automatic Claude reviews have been retired. To request a pull-request review, post a comment containing: Add |
|
Retrospective QA-fix tracking issue: #6387. It records the fix authored by @pruprakash and delivered by this merged PR. |
This branch was successfully deployed
4 participants
What does this PR do ?
Fixes Moonlight-16B training at PP=8/VP=1 and PP=4/VP=2 failing at config finalization, because both recipe pipeline layouts listed 31 decoder layers for a 27-layer model.
Changelog
src/megatron/bridge/recipes/moonlight/h100/moonlight_16b.py: re-split the (8, 1) and (4, 2) layouts as front-loaded [4, 4, 4, 3, 3, 3, 3, 3] (27 decoders).tests/unit_tests/recipes/test_moonlight_recipes.py: 1 test that runs every supported Moonlight layout through Megatron-CorePipelineParallelLayerLayout.validate_layer_layoutat the model depth.Additional Information
_get_moonlight_pipeline_layout(8, 1) and (4, 2) entries were4 + 4*6 + 3 = 31decoders; Moonlight-16B-A3B hasnum_hidden_layers=27and no MTP.nvcr.io/nvidian/nemo:nightly— REDNumber of decoder layers 31must match num_layers 27; GREEN 1 passed; full module 17 passed.