Skip to content

Commit 04c9ff7

Browse files
kamran-nvidiaclaude
andcommitted
refactor(model): drop Qwen3-local YaRN defaults now covered by the shared mapping
Qwen3 filled the Hugging Face YaRN defaults (beta_fast=32, beta_slow=1, truncate=True) locally because the shared mapping passed None for omitted keys. The shared GPT YaRN mapping now applies the same defaults, so the local copy is redundant. Qwen3's existing YaRN tests cover the shared path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Kamran Jafari <kjafarisadeg@nvidia.com>
1 parent 885765c commit 04c9ff7

1 file changed

Lines changed: 1 addition & 7 deletions

File tree

‎src/megatron/bridge/models/qwen/qwen3_bridge.py‎

Lines changed: 1 addition & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -59,14 +59,8 @@ def hf_config_to_model_config_kwargs(self, hf_config: PretrainedConfig) -> dict[
5959
rope_scaling=False,
6060
rope_scaling_factor=1.0,
6161
)
62-
# The shared mapping already selects "yarn" for YaRN-scaled checkpoints.
62+
# The shared mapping already selects "yarn" and its Hugging Face defaults for YaRN-scaled checkpoints.
6363
config_kwargs.setdefault("position_embedding_type", "rope")
64-
if config_kwargs["position_embedding_type"] == "yarn":
65-
# Qwen3 YaRN configs usually omit these keys; apply the Hugging Face defaults
66-
# instead of passing None to Megatron Core.
67-
config_kwargs.setdefault("yarn_beta_fast", 32.0)
68-
config_kwargs.setdefault("yarn_beta_slow", 1.0)
69-
config_kwargs.setdefault("yarn_correction_range_round_to_int", True)
7064
return config_kwargs
7165

7266
def hf_config_to_provider_kwargs(self, hf_config: PretrainedConfig) -> dict[str, Any]:

0 commit comments

Comments
 (0)