feat(peft): add moe_shared_loras export layout for shared-outer MoE LoRA

[HollowMan6]

What does this PR do ?

Add a moe_shared_loras option to adapter export that emits shared-outer routed-expert LoRA in the 2D experts.w1/w2/w3 layout consumed by vLLM's enable_moe_shared_loras mode.

Changelog

• Add moe_shared_loras: bool = False to AutoBridge.export_adapter_weights, MegatronPeftBridge.stream_adapter_weights_megatron_to_hf, and the registered stream_adapter_weights_megatron_to_hf dispatch.
• Add MegatronPeftBridge._stream_shared_outer_adapter_weights_2d_moe, which reuses the legacy shared-outer emission (mapping lookup, EP gather, fused gate/up split) and repacks it as:
  • experts.w1/w3.lora_A: shared gate/up factor, (1, r, hidden)
  • experts.w1/w3.lora_B: per-expert gate/up factors stacked in expert order, (E, intermediate, r)
  • experts.w2.lora_A: per-expert down factors, (E, r, intermediate)
  • experts.w2.lora_B: shared down factor, (1, hidden, r)
• Resolve the serving-layout flags by priority: stack_3d_moe > moe_shared_loras > expand_shared_outer. With all three unset, the legacy (SGLang) layout is unchanged.
• Raise a clear ValueError when moe_shared_loras meets non-gated experts, which have no w1/w3 pair.
• Forward moe_shared_loras (like with_megatron_names) only when enabled and check the bridge's streaming override up front, so bridges that override stream_adapter_weights_megatron_to_hf with an explicit signature keep working by default. _check_with_megatron_names_support is generalized to _check_stream_kwarg_support.
• Docs: document the three shared-outer export layouts and their priority in docs/training/peft.md.
• Tests: value-level checks of the w1/w2/w3 layout (w1 = gate, w3 = up, expert order, shared factors at expert-dim 1), flag priority, the non-gated error path, and AutoBridge forwarding / legacy-override compatibility.

GitHub Actions CI

See the CI section in the Contributing doc for how to trigger the CI. A Nvidia developer will need to approve and trigger the CI for external contributors.

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you add or update any necessary documentation?
• Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  • Reviewer: Does the PR have correct import guards for all optional libraries?

If you haven't finished some of the above items you can still open "Draft" PR.

Additional Information

• Follow-up to feat(peft): expand_shared_outer export flag for shared-outer MoE LoRA #5588 (expand_shared_outer) and Support 3D-MoE LoRA export for shared-outer routed-expert adapters #5939 (stack_3d_moe). The new flag is additive and defaults to False, so existing export output is unchanged.
• Tested: python -m pytest tests/unit_tests/models/test_adapter_export.py tests/unit_tests/models/test_auto_bridge.py tests/unit_tests/models/test_model_bridge_lora.py tests/unit_tests/models/megatron_mimo/conversion/test_orchestrator.py (305 passed).

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.