Skip to content

Commit e0a9b4a

Browse files
committed
chore(skills): deduplicate expert overlap benchmark
Signed-off-by: Chen Cui <chcui@nvidia.com>
1 parent c03966a commit e0a9b4a

1 file changed

Lines changed: 4 additions & 19 deletions

File tree

  • skills/nemo-mbridge-perf-expert-parallel-overlap

‎skills/nemo-mbridge-perf-expert-parallel-overlap/SKILL.md‎

Lines changed: 4 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -84,8 +84,10 @@ Steady-window timing from that benchmark:
8484
| EP overlap | 31.31s | 1.317x |
8585
| EP overlap plus `delay_wgrad_compute` | 31.20s | 1.322x |
8686

87-
This shows a clear plain-overlap win, but no meaningful independent gain from
88-
delayed wgrad in that benchmark.
87+
This is evidence for enabling plain EP overlap on this inter-node all-to-all
88+
shape. It does not show a meaningful independent win from delayed wgrad, and it
89+
does not validate fused MoE permutation because that path was disabled for the
90+
runtime stack.
8991

9092
## Enablement
9193

@@ -156,23 +158,6 @@ Use this as the correctness-first starting point. Add delayed wgrad, flex
156158
dispatch, and CUDA-graph interactions only after the plain overlap path is
157159
known to work.
158160

159-
## Measured Short-Run Evidence
160-
161-
A 2026-05-18 current-main H100 x16 Qwen3 30B-A3B mock pretraining run used
162-
`EP=16`, `alltoall`, BF16, global batch size 1024, CUDA graphs disabled, and
163-
`moe_permute_fusion=false`. With iterations 3-8 as the steady window:
164-
165-
| Case | Steady mean | Relative |
166-
|---|---:|---:|
167-
| no EP overlap | 41.25s | 1.000x |
168-
| EP overlap | 31.31s | 1.317x |
169-
| EP overlap plus `delay_wgrad_compute` | 31.20s | 1.322x |
170-
171-
This is evidence for enabling plain EP overlap on this inter-node all-to-all
172-
shape. It does not show a meaningful independent win from delayed wgrad, and it
173-
does not validate fused MoE permutation because that path was disabled for the
174-
runtime stack.
175-
176161
## Minimal Runnable Command
177162

178163
Performance harness example inside a Slurm allocation. Keep the model,

0 commit comments

Comments
 (0)