Skip to content

Commit 368659c

Browse files
authored
Merge branch 'main' into dpo_support
2 parents 28dc38c + 912c16e commit 368659c

4 files changed

Lines changed: 75 additions & 62 deletions

File tree

‎.main.commit‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
8949def81eb744b9219e2d794c80265928c40a5b
1+
fde43b6d617752433310b4668c6b5c55e503521a

‎3rdparty/Megatron-LM‎

Submodule Megatron-LM updated 65 files

‎tests/unit_tests/models/qwen_vl/modelling_qwen3_vl/test_text_model_forward.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -254,6 +254,8 @@ def fake_output_layer(hidden, weight=None, runtime_gather_output=None): # noqa:
254254
mtp_loss_scaling_factor=1.0,
255255
calculate_per_token_loss=True,
256256
use_mup=False,
257+
# process_mtp_loss reads the CP layout; mirror TransformerConfig's default.
258+
attention_cp_layout="zigzag",
257259
)
258260
output = process_mtp_loss(
259261
hidden_states=hidden_states,

0 commit comments

Comments
 (0)