"ground_truth": "The answer should use the expert-parallel overlap skill and focus on the plain alltoall benchmark. It should state the benchmark shape: Qwen3 MoE 30B-A3B SFT, 16 H100 GPUs, EP=16, alltoall, BF16, global batch size 1024, CUDA graphs disabled, moe_permute_fusion=false, with iterations 3-8 as the steady window. It should enable plain EP overlap with --cuda_graph_impl none, --moe_flex_dispatcher_backend None, --moe_a2a_overlap false, comm_overlap.overlap_moe_expert_parallel_comm=true, comm_overlap.delay_wgrad_compute=false, and model.moe_shared_expert_overlap=false. It should warn not to use --moe_a2a_overlap true for this isolation test because the helper enables both overlap_moe_expert_parallel_comm and delay_wgrad_compute. It should quote the timing comparison: no EP overlap 41.25s (1.000x), EP overlap 31.31s (1.317x), EP overlap plus delay_wgrad_compute 31.20s (1.322x), and say delayed wgrad did not show a meaningful independent win in this benchmark.",
0 commit comments