Skip to content

refactor(model): build Hy V3 and MiniMax-M2 from builder-backed model configs - #6392

Open
kamran-nvidia wants to merge 3 commits into
mainfrom
kjafarisadeg/hyv3-minimaxm2-model-config
Open

kamran-nvidia wants to merge 3 commits into
mainfrom
kjafarisadeg/hyv3-minimaxm2-model-config

Conversation

@kamran-nvidia

@kamran-nvidia kamran-nvidia commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do ?

Moves Hy V3 and MiniMax-M2 onto the builder-backed ModelConfig / ModelBuilder path for checkpoint conversion, following Muse Glimmer. Neither family has library recipes.

Changelog

  • Opt-in after fix(model): default MODEL_CONFIG_CLASS to None and opt in per family #6305: MODEL_CONFIG_CLASS now defaults to None, so each migrated bridge sets MODEL_CONFIG_CLASS = BridgeGPTModelConfig next to USE_MODEL_CONFIG_FOR_CONVERSION = True (as Qwen3 and Llama do), and its tests assert it.
  • HYV3Bridge: provider_bridge() is replaced by hf_config_to_model_config_kwargs(), and the provider path reuses the same kwargs. The builder config leaves transformer_layer_spec unset because MCore's builder default for MoE models is already get_gpt_decoder_block_spec, which honors the dense-first moe_layer_freq. The deprecated provider path keeps its explicit decoder block spec in hf_config_to_provider_kwargs(). The forced bf16 dtype fields carry over. USE_MODEL_CONFIG_FOR_CONVERSION = True.
  • MiniMaxM2Bridge: same kwargs migration. The builder config carries transformer_layer_spec=minimax_m2_layer_spec; that spec reads only fields the builder config resolves (num_moe_experts, moe_grouped_gemm), so both paths build the full-dimension QK norm. rotary_dim → rotary_percent handling is unchanged. USE_MODEL_CONFIG_FOR_CONVERSION = True.
  • Tests:
    • New tests/unit_tests/models/hy_v3/; Hy V3 had no bridge unit tests. It covers the builder mapping, provider/builder parity for bf16 and fp16 sources, and megatron_to_hf_config from a builder config.
    • MiniMax-M2 covers the builder mapping, parity with and without rotary_dim, and layer-spec equivalence between provider and builder config.

Verification

Each family was validated separately on tiny random-weight checkpoints (single GB200 node, NeMo 26.10 container, transformers 5.15 for both HF models).

Conversion paths:

  • scripts/conversion GPU import at TP2 and export.
  • AutoBridge.import_ckpt / export_ckpt.
  • Load + forward as in compare.py.
  • roundtrip_checkpoint.

Each was run on this branch and on main.

Check Hy V3 (dense-first MoE, shared expert) MiniMax-M2 (sigmoid MoE, full-dim QK norm)
Provider fields, main vs this branch identical (bf16 and fp16 sources) identical (incl. rotary_dim)
get_model_config() vs provider 0 diffs (19 / 22 on main) 0 diffs (14 / 15 on main)
Export vs source HF weights (GPU TP2, CPU, and from a main-imported checkpoint) bit-exact except expert-bias dtype (see below) bit-exact except expert-bias dtype (see below)
Logits: this branch vs main, and this branch on a main-imported checkpoint bitwise-identical bitwise-identical
Logits vs HF cosine 0.999987 cosine 0.999914
roundtrip_checkpoint TP2 pass pass
Unit tests (family run) 7 passed 36 passed

Regression: 334 tests (1 skipped) across models/hy_v3, models/minimax_m2, test_auto_bridge.py, test_gpt_model_config.py, the AutoBridge registration matrix, conversion/ and doc_consistency.

Lint: ruff check and ruff format --check are clean (ruff 0.16.7; pre-commit itself not run).

Known, pre-existing: a bf16 source checkpoint exports the router expert bias (mlp.expert_bias / e_score_correction_bias) as fp32, because Megatron keeps it in fp32. The values are identical after a cast, and main's own code exports these checkpoints the same way (verified for both families).

After rebasing onto main with #6305 (Megatron Core a98aed553): 215 passed, 1 skipped, including #6305's all-bridges opt-in check in test_gpt_model_config.py; ruff clean.

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you add or update any necessary documentation?
  • Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
    • Reviewer: Does the PR have correct import guards for all optional libraries?

Additional Information

🤖 Generated with Claude Code

@kamran-nvidia kamran-nvidia added feature New capabilities, enhancements, or enablement work area:model Model implementations and HF bridge logic labels Oct 9, 2026
@copy-pr-bot

copy-pr-bot Bot commented Oct 9, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@kamran-nvidia

Copy link
Copy Markdown
Contributor Author

/ok to test d44b78b

kamran-nvidia and others added 3 commits October 14, 2026 06:57
Move Hy V3 onto the ModelConfig/ModelBuilder path, following Muse Glimmer:

- HYV3Bridge maps Hugging Face configs in hf_config_to_model_config_kwargs().
  The builder's default MoE layer spec is the GPT decoder block spec, which
  honors the dense-first moe_layer_freq; the deprecated provider path keeps
  its explicit decoder block spec. USE_MODEL_CONFIG_FOR_CONVERSION = True.
- Add Hy V3 bridge unit tests (none existed).

Signed-off-by: Kamran Jafari <kjafarisadeg@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Move MiniMax-M2 onto the ModelConfig/ModelBuilder path, following Muse
Glimmer. MiniMaxM2Bridge maps Hugging Face configs in
hf_config_to_model_config_kwargs(), including the full-dimension QK-norm
layer spec, and the deprecated provider path reuses the same kwargs.
USE_MODEL_CONFIG_FOR_CONVERSION = True.

Signed-off-by: Kamran Jafari <kjafarisadeg@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…configs

After #6305, MODEL_CONFIG_CLASS defaults to None and each family opts in.
Set MODEL_CONFIG_CLASS = BridgeGPTModelConfig on the bridges this PR
migrates, matching Qwen3 and Llama, and assert it in their tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Kamran Jafari <kjafarisadeg@nvidia.com>
@kamran-nvidia

Copy link
Copy Markdown
Contributor Author

/ok to test 3d8a518

@github-actions

Copy link
Copy Markdown
Contributor

Automatic Claude reviews have been retired. To request a pull-request review, post a comment containing:

/review

Add model=claude to use a Claude reviewer (the default is model=codex). mode=light|strict selects the review depth; for example, /review model=claude mode=strict. Comment /review help for all options.

This branch was successfully deployed

1 active deployment
test — 3d8a5183 Deployed Oct 14, 2026 by copy-pr-bot[bot] via cicd-wait-in-queue #22857
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:model Model implementations and HF bridge logic feature New capabilities, enhancements, or enablement work needs-review PR is ready for code review and waiting on a reviewer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants