[training] fix: restore alltoall when flex backend is cleared

[yaoyu-33]

Summary

• restore the standard alltoall dispatcher when a finalized flex backend is cleared
• cover the post-factory override lifecycle with a focused regression test

Supported trigger and impact

The shared recipe runner accepts trailing configuration overrides after the recipe factory returns. A flex-enabled MoE recipe such as Qwen3-Next therefore accepts model.moe_flex_dispatcher_backend=null after the factory has selected the flex dispatcher.

Previously that override cleared only the backend. Bridge validation accepted the resulting flex dispatcher with no backend, and pinned Megatron-Core then raised ValueError: Invalid backend: None while constructing the MoE layer. This prevents an otherwise documented alltoall configuration from starting training.

This is a recurrence of the invariant documented in #2861. Its earlier fix was scoped to the legacy performance launcher; the unified recipe finalization path remained exposed.

Root cause and fix

ConfigContainer calls validate_flex_dispatcher_backend after post-factory overrides and model finalization. That validation checked hardware support for named DeepEP and HybridEP backends but did not reconcile flex with a cleared backend.

The fix uses the existing alltoall fallback at this final shared boundary before Megatron-Core model construction. It does not change backend hardware support or enable any new mode.

Regression evidence

Fail before:

• uv run --no-sync python -m pytest tests/unit_tests/training/test_deepep.py::TestFlexDispatcherFallback::test_none_backend_override_falls_back_to_alltoall -q
• 1 failed: dispatcher remained flex instead of alltoall after the backend was cleared

Pass after:

• same focused command: 1 passed
• uv run --no-sync python -m pytest tests/unit_tests/training/test_deepep.py -q: 18 passed
• uv run --no-sync python -m pytest tests/unit_tests/recipes/test_qwen_recipes.py -k qwen3_next -q: 3 passed, 85 deselected
• uv run --no-sync pre-commit run --all-files: passed
• uv run --no-sync mypy --strict --ignore-missing-imports src/megatron/bridge/training/flex_dispatcher_backend.py: passed

Scope

No Megatron-Core changes, public API changes, dependency changes, CI changes, or backend availability changes are included.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[yaoyu-33]

/ok to test dd3d79b

[claude]

LGTM. The added guard in validate_flex_dispatcher_backend closes a gap where a flex dispatcher type with a cleared backend previously passed validation; it now falls back to alltoall. The new test covers this override path.

Suggested test cases:

• test_none_backend_override_falls_back_to_alltoall
• No perf tests impacted.