You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 995ef0c
Browse filesBrowse the repository at this point in the historyBrowse files
Copy file name to clipboardExpand all lines: skills/nemo-mbridge-perf-activation-recompute/SKILL.md
+9-10Lines changed: 9 additions & 10 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -19,8 +19,8 @@ Activation recompute (activation checkpointing) trades additional forward work d
19
19
## Quick Decision Guide
20
20
21
21
1. Confirm the pressure is real allocation, not allocator fragmentation. Compare `max_memory_allocated()` with `max_memory_reserved()` on every rank.
22
-
2.Start final library, performance, and verification-card recipes with no recompute when the workload fits. Keep an explicit no-recompute control. Under selective granularity, `recompute_modules=[]` is valid and useful for this comparison.
23
-
3.When no recompute does not fit, select the narrowest supported boundary from the architecture and observed peak:
22
+
2.For final library, performance, and verification-card recipes, no recompute and selective recompute on cheap supported modules are both appropriate starting choices. Choose by measured memory headroom and steady-state performance; a no-recompute OOM is not a prerequisite for selective recompute. Keep an explicit no-recompute control when it fits. Under selective granularity, `recompute_modules=[]` is valid and useful for this comparison.
23
+
3.For selective recompute, start with cheap supported boundaries selected from the architecture and observed peak:
24
24
-**Standard attention:**`core_attn` is the common first candidate. It is strongest when unfused attention materializes score/probability tensors. With Transformer Engine fused or Flash Attention, compare it against `[]` because those backends already rematerialize attention internals.
25
25
-**Multi-Latent Attention (MLA):** start with `mla_up_proj` when expanded Q/K/V projections dominate. Add `core_attn` only when the attention-core state still matters.
26
26
-**Grouped MoE:** start with `moe_act` when the expert intermediate activation dominates; add `layernorm` when norm outputs are material. Use whole `moe` recompute only after accounting for the extra expert compute and communication it replays.
@@ -138,8 +138,8 @@ iteration 2. Time and throughput are means over iterations 11--20.
0 commit comments