[recipe] fix: allow MoE recipe construction without CUDA

[yaoyu-33]

What does this PR do?

Public Qwen3-MoE SFT and PEFT recipes enable a flex dispatcher while they are being constructed. The official offline-packing workflow constructs these recipes before a GPU training job, but apply_flex_dispatcher_backend() unconditionally queried CUDA device 0. On a CPU preprocessing host this raised during recipe construction, before packing could start.

The changed path is:

prepare_gpt_sft_packed_data.py -> public Qwen3-MoE recipe factory -> apply_flex_dispatcher_backend() -> torch.cuda.get_device_properties(0)

The helper now records the requested supported flex backend without probing device properties when CUDA is unavailable. GPU processes keep the existing architecture check, and CUDA probe errors still propagate when CUDA is reported available. Runtime validation is unchanged.

Scope is limited to CPU-side construction of supported DeepEP and HybridEP recipe configs. It does not make those backends executable without GPUs and does not change training-time hardware validation.

Regression evidence

The focused regression constructs the public qwen3_30b_a3b_peft_config with a real MoE-shaped provider and asserts that CUDA device properties are not queried when CUDA is unavailable.

Applied as a test-only change to 2e77041c194d106beb7462e226d7ca06b33ea63f:

uv run python -m pytest tests/unit_tests/recipes/test_qwen_recipes.py::test_qwen3_moe_peft_recipe_builds_without_cuda -q

Result: failed at torch.cuda.get_device_properties(0) with recipe construction probed CUDA device properties.

With this fix, the unchanged command passes: 1 passed.

Validation

uv run python -m pytest tests/unit_tests/training/test_deepep.py -q
# 18 passed

uv run python -m pytest tests/unit_tests/recipes/test_qwen_recipes.py -k 'qwen3_moe or qwen3_30b_a3b' -q
# 21 passed, 68 deselected

git diff --check
# passed

uv run pre-commit run --all-files
# passed

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[yaoyu-33]

/ok to test 962083c

[claude]

Light review - LGTM.

Clean, well-scoped fix: apply_flex_dispatcher_backend now short-circuits when CUDA is unavailable and the backend is deepep/hybridep, setting the flex config without probing device properties. This lets offline packing build MoE recipes (e.g. qwen3_30b_a3b_peft_config) on a CPU-only host. Arch validation is correctly deferred to validate_flex_dispatcher_backend, which still runs at runtime.

Observations:

• Logic is correct - the CUDA guard is placed before the get_device_properties(0) call, so no GPU probe happens on CPU. The blank-line removal is cosmetic only.
• Test coverage is good: the _mock_cuda_available autouse fixture preserves existing test_deepep.py behavior; test_apply_flex_dispatcher_backend_preserves_cuda_probe_errors confirms real CUDA init errors still propagate; test_qwen3_moe_peft_recipe_builds_without_cuda asserts the recipe builds and get_device_properties is never called.
• One optional note: add a comment pointing to validate_flex_dispatcher_backend so a future reader knows the arch gate is deferred, not dropped.

Suggested test cases

• test_apply_flex_dispatcher_backend_preserves_cuda_probe_errors (added)
• test_qwen3_moe_peft_recipe_builds_without_cuda (added)
• Consider also: a CPU/no-CUDA case for moe_flex_dispatcher_backend=hybridep to mirror the deepep recipe test
• Consider also: a CPU/no-CUDA case with an invalid backend name to confirm it still hits the 'Not a valid flex dispatcher backend' warning path rather than the new short-circuit

No perf tests impacted.

[claude]

This CPU/no-CUDA branch intentionally skips the GPU arch check and sets flex/backend directly. That's fine because validate_flex_dispatcher_backend re-checks the arch at runtime — but the safety of this path depends on that validator always running before the flex dispatcher is actually used. Consider a brief comment here pointing to validate_flex_dispatcher_backend so a future reader knows the arch gate is deferred, not dropped.