Skip to content

Commit d3af42b

Browse files
committed
test(model): drop MTP from the EXAONE MoE toy checkpoint
ExaoneMoeForCausalLM declares _keys_to_ignore_on_load_unexpected = [r"mtp.*"], so released checkpoints carry MTP weights but the modelling class does not build them. A toy constructed from that class with num_nextn_predict_layers set therefore saves no MTP weights, while the provider still builds MTP layers and the bridge still registers the MTP mappings off the same config value. Those layers were being compared without ever having been loaded. The new mapping validation reports them as missing instead, which is correct. Drop MTP from the toy so it represents something the class can actually produce. Verified with the roundtrip the functional test drives, TP=2: 89/89 weights matched. Signed-off-by: Minho Ryu <ryumin93@gmail.com>
1 parent b6a6612 commit d3af42b

1 file changed

Lines changed: 5 additions & 1 deletion

File tree

‎tests/functional_tests/test_groups/models/exaone/test_exaone_conversion.py‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -100,7 +100,11 @@
100100
"norm_topk_prob": True,
101101
"routed_scaling_factor": 2.5,
102102
"scoring_func": "sigmoid",
103-
"num_nextn_predict_layers": 1,
103+
# No MTP here. `ExaoneMoeForCausalLM` declares `_keys_to_ignore_on_load_unexpected =
104+
# [r"mtp.*"]`, so released checkpoints carry MTP weights but the modelling class does
105+
# not build them. A toy constructed from the class therefore saves nothing for the MTP
106+
# mappings the bridge registers off `num_nextn_predict_layers`, while the provider still
107+
# builds MTP layers, and the roundtrip would compare layers that were never loaded.
104108
}
105109

106110

0 commit comments

Comments
 (0)