Skip to content

Commit 50fc395

Browse files
committed
[recipe] fix: remove redundant gradient_accumulation_fusion override in NT3 Nano
Now that the checker correctly returns True when transformer_engine.pytorch is available, MambaModelProvider's default_factory handles this correctly. The explicit override is no longer needed. Signed-off-by: Yu Yao <yaoyu.094@gmail.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
1 parent f86d0a8 commit 50fc395

1 file changed

Lines changed: 0 additions & 3 deletions

File tree

‎src/megatron/bridge/recipes/nemotronh/nemotron_3_nano.py‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -187,7 +187,6 @@ def nemotron_3_nano_pretrain_config() -> ConfigContainer:
187187

188188
cfg.model.init_method_std = 0.0173
189189
cfg.model.apply_rope_fusion = False
190-
cfg.model.gradient_accumulation_fusion = True
191190
cfg.model.use_fused_weighted_squared_relu = True
192191

193192
return cfg
@@ -300,7 +299,6 @@ def nemotron_3_nano_sft_config() -> ConfigContainer:
300299
cfg.model.moe_grouped_gemm = True
301300
cfg.model.cross_entropy_loss_fusion = True
302301
cfg.model.cross_entropy_fusion_impl = "native"
303-
cfg.model.gradient_accumulation_fusion = True
304302

305303
# Memory saving (recompute & offloading)
306304
cfg.model.recompute_granularity = None
@@ -490,7 +488,6 @@ def nemotron_3_nano_peft_config(
490488
cfg.model.moe_grouped_gemm = True
491489
cfg.model.cross_entropy_loss_fusion = True
492490
cfg.model.cross_entropy_fusion_impl = "native"
493-
cfg.model.gradient_accumulation_fusion = True
494491

495492
# Memory saving
496493
cfg.model.recompute_granularity = None

0 commit comments

Comments
 (0)