Commit 9a19a74
committed
[misc] say why the collective entry was dropped, not just that it was
The replaced set's _c10d_functional::reduce_scatter_tensor is correct where it
came from: torchtitan runs Megatron-style sequence parallelism, which
reduce-scatters inside the forward, and compiles, which is what puts collectives
in the _c10d_functional namespace. FastVideo redistributes with Ulysses
all-to-all and cannot compile the training path, so neither condition holds.
Verified against FSDP2 with the trainer's wrapping order: re-adding both
collective names leaves every op's MUST_SAVE/PREFER_RECOMPUTE decision unchanged
and the loss bitwise identical, so the removal is a no-op.1 parent 695c716 commit 9a19a74
1 file changed
Lines changed: 8 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
53 | 53 | | |
54 | 54 | | |
55 | 55 | | |
56 | | - | |
57 | | - | |
58 | | - | |
59 | | - | |
60 | | - | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
61 | 64 | | |
62 | 65 | | |
63 | 66 | | |
| |||
0 commit comments