Skip to content

Commit 1986b1f

Browse files
committed
[docs] attn_only: note it only saves attention for dispatcher-visible backends
The MUST_SAVE policy runs via __torch_dispatch__, so it only sees the attention forward when it is a registered op (torch SDPA, FA4/cute). The flash_attn-library FA2/FA3 training forward routes to the raw external func (opaque to SAC), so its attention is recomputed — correct, but no speedup over FULL for that backend. Document the caveat; no behavior change.
1 parent 8e18bc3 commit 1986b1f

1 file changed

Lines changed: 5 additions & 1 deletion

File tree

fastvideo/training/activation_checkpoint.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -130,7 +130,11 @@ def _apply_activation_checkpointing_attn_only(module: torch.nn.Module) -> torch.
130130
intermediates are huge but cheap to recompute — saving them is what makes
131131
the stock 'ops' mode OOM). Eliminates the attention forward recompute that
132132
FULL mode pays (flash_fwd runs 2x under FULL) while staying within memory.
133-
Orthogonal to torch.compile."""
133+
Orthogonal to torch.compile.
134+
135+
Backend caveat: only saves the attention forward for dispatcher-visible ops
136+
(SDPA, FA4/cute); the flash_attn-lib FA2/FA3 forward is opaque to SAC and is
137+
recomputed (correct, no speedup)."""
134138
from torch.utils.checkpoint import (CheckpointPolicy, create_selective_checkpoint_contexts)
135139

136140
def _attn_only_policy(ctx, func, *args, **kwargs):

0 commit comments

Comments
 (0)