Repository navigation
Commit a6692f3
committed
[model] fix: restore explicit gradient_accumulation_fusion=True overrides
The dynamic checker (can_enable_gradient_accumulation_fusion) is
unreliable in the training environment — it returns False even when the
fused_weight_gradient_mlp_cuda extension is available at runtime.
Restore explicit True overrides in DSV2, DSV3, Kimi bridges and NT3
Nano recipe to guarantee the fusion is always enabled for these models.
Signed-off-by: Yu Yao <yaoyu.094@gmail.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>1 parent 41520f3 commit a6692f3
4 files changed
Lines changed: 6 additions & 0 deletions
File tree
- src/megatron/bridge
- models
- deepseek
- kimi
- recipes/nemotronh
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
62 | 62 | | |
63 | 63 | | |
64 | 64 | | |
| 65 | + | |
65 | 66 | | |
66 | 67 | | |
67 | 68 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
69 | 69 | | |
70 | 70 | | |
71 | 71 | | |
| 72 | + | |
72 | 73 | | |
73 | 74 | | |
74 | 75 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
64 | 64 | | |
65 | 65 | | |
66 | 66 | | |
| 67 | + | |
67 | 68 | | |
68 | 69 | | |
69 | 70 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
187 | 187 | | |
188 | 188 | | |
189 | 189 | | |
| 190 | + | |
190 | 191 | | |
191 | 192 | | |
192 | 193 | | |
| |||
299 | 300 | | |
300 | 301 | | |
301 | 302 | | |
| 303 | + | |
302 | 304 | | |
303 | 305 | | |
304 | 306 | | |
| |||
488 | 490 | | |
489 | 491 | | |
490 | 492 | | |
| 493 | + | |
491 | 494 | | |
492 | 495 | | |
493 | 496 | | |
| |||
0 commit comments