refactor(model): build Ling MoE V2 and EXAONE 4.0 from builder-backed model configs

[kamran-nvidia]

What does this PR do ?

Moves Ling MoE V2 (Bailing) and EXAONE 4.0 onto the builder-backed ModelConfig / ModelBuilder path, for recipes and checkpoint conversion, following Muse Glimmer.

Changelog

• Opt-in after fix(model): default MODEL_CONFIG_CLASS to None and opt in per family #6305: MODEL_CONFIG_CLASS now defaults to None, so each migrated bridge sets MODEL_CONFIG_CLASS = BridgeGPTModelConfig next to USE_MODEL_CONFIG_FOR_CONVERSION = True (as Qwen3 and Llama do), and its tests assert it.
• BailingMoeV2Bridge: provider_bridge() is replaced by hf_config_to_model_config_kwargs(), and the provider path reuses the same kwargs. The builder config leaves transformer_layer_spec unset because MCore's builder default for MoE models is already get_gpt_decoder_block_spec, which honors the dense-first moe_layer_freq. The deprecated provider path keeps its explicit decoder block spec in hf_config_to_provider_kwargs(). USE_MODEL_CONFIG_FOR_CONVERSION = True.
• Exaone4Bridge: same kwargs migration. The builder config carries the Post-LN exaone4_layer_spec, which does not read its config. USE_MODEL_CONFIG_FOR_CONVERSION = True. The EXAONE 4.0 recipes use get_model_config().
• EXAONE 4.0 llama3 RoPE scaling — one intentional behavior change:
  • Before: the bridge stored low_freq_factor, high_freq_factor and original_max_position_embeddings as undeclared attributes on the provider. Megatron Core never reads them: RotaryEmbedding calls _apply_scaling(inv_freq, factor=rope_scaling_factor) with fixed 1 / 4 / 8192. They were only echoed back on export, and a strict builder config cannot carry undeclared fields.
  • Now: only rope_scaling and rope_scaling_factor are mapped. Export writes Megatron Core's fixed values (as LlamaBridge does), and a warning is logged when a checkpoint's values differ from what Megatron applies.
  • Impact: EXAONE 4.0 checkpoints use exactly 1 / 4 / 8192, so their behavior is unchanged.
• Tests:
  • Bailing: builder mapping and provider/builder parity, with and without QKV bias.
  • EXAONE: builder mapping, parity, export of RoPE values from a builder config, and the non-default-value warning.
  • New strict recipe test for the three EXAONE 4.0 recipes.

Verification

Each family was validated separately on tiny random-weight checkpoints (single GB200 node, NeMo 26.10 container; Bailing uses the HF modeling code vendored in this repo, EXAONE uses transformers 5.15).

Conversion paths:

• scripts/conversion GPU import at TP2 and export.
• AutoBridge.import_ckpt / export_ckpt.
• Load + forward as in compare.py.
• roundtrip_checkpoint.

Each was run on this branch and on main.

Check	Ling MoE V2 (grouped sigmoid routing, dense-first)	EXAONE 4.0 (Post-LN, llama3 RoPE, tied embeddings)
Provider fields, main vs this branch	identical (incl. QKV bias)	identical (llama3 and no scaling)
get_model_config() vs provider	0 diffs (17 / 18 on main)	0 diffs (10 / 9 on main)
Export vs source HF weights (GPU TP2, CPU, and from a main-imported checkpoint)	bit-exact except expert-bias dtype (see below)	bit-exact
Logits: this branch vs main, and this branch on a main-imported checkpoint	bitwise-identical	bitwise-identical
roundtrip_checkpoint TP2	pass	pass
Unit tests (family run)	6 passed	15 passed

Regression: 11 Bailing/EXAONE recipe factories and 312 tests (1 skipped) across models/bailing, models/exaone, the EXAONE 4.0 recipe test, test_auto_bridge.py, test_gpt_model_config.py, the AutoBridge registration matrix, conversion/ and doc_consistency.

Lint: ruff check and ruff format --check are clean (ruff 0.16.7; pre-commit itself not run).

Pre-existing, identical on main; not changed by this PR:

• Expert-bias export dtype (Ling MoE V2): a bf16 source exports the router expert bias as fp32 because Megatron keeps it in fp32. Values are identical after a cast.
• Ling MoE V2 vs HF logits: cosine about 0.97 for the tiny model, and main gives the same value. The HF-side value varies slightly between runs while the Megatron logits are bit-identical.
• EXAONE 4.0 vs HF (transformers 5.15) logits: cosine about 0.73, on main too. It persists with all-global attention and with no RoPE scaling, so it is structural rather than RoPE- or sliding-window-related.
• EXAONE 4.0 unsupported shapes: the bridge maps tied embeddings only (no lm_head mapping), and HF's hybrid sliding-window attention (global layers without RoPE) is not modeled.

These are worth separate investigation against real checkpoints.

After rebasing onto main with #6305 (Megatron Core a98aed553): 193 passed, 1 skipped, including #6305's all-bridges opt-in check in test_gpt_model_config.py; ruff clean.

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you add or update any necessary documentation?
• Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  • Reviewer: Does the PR have correct import guards for all optional libraries?

Additional Information

• Part of the incremental provider → builder migration (refactor(model): add Qwen3 builder-backed model config with YaRN support #6365, refactor(model): build Qwen2 and MiMo from builder-backed model configs #6379, refactor(model): build OLMoE and GLM-4.5 from builder-backed model configs #6383, refactor(model): build Hy V3 and MiniMax-M2 from builder-backed model configs #6392; two families per PR).
• refactor(model)!: remove retired model support #6370 only rewrites comment and docstring wording in exaone4_bridge.py; the overlap with this PR is a trivial docstring merge.
• Targets main directly.
• Tied checkpoints at PP > 1 through the builder path need the shared fix in fix(ckpt): resolve tied embeddings for builder-backed models at PP > 1 #6460 (raised by @cuichenx); this PR will be rebased onto it once it merges.

🤖 Generated with Claude Code

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[kamran-nvidia]

/ok to test e0d514e

[cuichenx]

One concern with tied-weight conversion when enabling the builder path; details inline.

[cuichenx]

Tied embeddings with PP > 1 lose the conversion tie flag

Enabling the builder path here puts share_embeddings_and_output_weights on the outer BridgeGPTModelConfig, while the runtime model's .config is only the nested TransformerConfig. build_conversion_tasks() reads the nested config, and _share_embeddings_and_output_weights() defaults the missing field to False.

For tied EXAONE at PP > 1, the last pipeline stage materializes output_layer.weight. It is consequently no longer filtered from conversion, but EXAONE's mapping registry has no mapping for this parameter, so conversion fails. The post-import shared-embedding broadcast also reads the missing flag and is skipped. Previously the provider's .config carried the flag. PP=1 avoids allocating the duplicate output parameter, so the reported TP2 checks do not cover this case.

Could we fix tie-flag resolution in the shared conversion helpers and add a tied EXAONE PP2 import/export test, including output-stage weights and synchronization? The same issue is noted on #6379, so one shared fix should cover both. HF initialization for training via load_hf_weights() also reaches this path.

This is separate from the pre-existing EXAONE architecture/parity limitations listed in the PR. The finding is based on pinned source inspection and source-level probes; I have not run a distributed checkpoint reproduction.

[kamran-nvidia]

Thanks, confirmed. A tied tiny EXAONE 4.0 at PP=2 through the builder path fails on this branch with ValueError: No mapping found for the following Megatron parameter(s): output_layer.weight.

One shared fix covers this and #6379: #6460 makes _share_embeddings_and_output_weights() fall back to the flag on the built language model when the nested TransformerConfig doesn't declare it. With it, the last-stage output_layer.weight equals the HF embed_tokens, and the export reproduces all 46 HF tensors exactly with no lm_head.weight. #6460 adds a tied PP=2 builder-path functional test (Qwen3, since its toy checkpoint is tied and already in L0). I'll rebase this PR onto it once it merges.

[kamran-nvidia]

/ok to test 8ceb812

[github-actions]

Automatic Claude reviews have been retired. To request a pull-request review, post a comment containing:

/review

Add model=claude to use a Claude reviewer (the default is model=codex). mode=light|strict selects the review depth; for example, /review model=claude mode=strict. Comment /review help for all options.