Skip to content

Commit 115309e

Browse files
committed
refactor(recipes): normalize h100 peft recipe aliases
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
1 parent 1a932f3 commit 115309e

89 files changed

Lines changed: 2734 additions & 2556 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎docs/fern/versions/nightly/pages/recipe-usage.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ This guide will cover the next steps to make use of a training recipe, including
1212

1313
## Choosing a recipe or a new config
1414

15-
Start from an exported recipe when the model family and workflow already exist in `megatron.bridge.recipes`. Hardware-specific H100 recipes such as `llama3_8b_pretrain_2gpu_h100_bf16_config`, `llama32_1b_sft_1gpu_h100_bf16_config`, and `qwen3_8b_lora_1gpu_h100_bf16_config` are the canonical recipe names and share the same `<model>_<task>_<num>gpu_<gpu>_<dtype>_config` naming strategy as performance recipes. The older recipe names remain available as compatibility wrappers. Override those defaults for your dataset, checkpoint paths, run length, parallelism, or precision before creating a new recipe.
15+
Start from an exported recipe when the model family and workflow already exist in `megatron.bridge.recipes`. Hardware-specific H100 recipes such as `llama3_8b_pretrain_2gpu_h100_bf16_config`, `llama32_1b_sft_1gpu_h100_bf16_config`, and `qwen3_8b_peft_1gpu_h100_bf16_config` are the canonical recipe names and share the same `<model>_<task>_<num>gpu_<gpu>_<dtype>_config` naming strategy as performance recipes. The older recipe names remain available as compatibility wrappers. Override those defaults for your dataset, checkpoint paths, run length, parallelism, or precision before creating a new recipe.
1616

1717
Create a new recipe or config when the base model architecture is not represented by an existing model provider, the checkpoint conversion needs a new bridge, the forward step or dataset provider is model-specific, or you need a reusable configuration that will be shared across jobs. If the Hugging Face model is already supported by `AutoBridge`, you usually only need to start from the closest recipe and override the model provider or `hf_path`.
1818

‎docs/fern/versions/nightly/pages/training/data-preparation.mdx‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -115,7 +115,7 @@ The generic launcher provides preset Hugging Face text datasets through `--datas
115115

116116
```bash
117117
uv run python -m torch.distributed.run --nproc_per_node=1 scripts/training/run_recipe.py \
118-
--recipe llama32_1b_lora_1gpu_h100_bf16_config \
118+
--recipe llama32_1b_peft_1gpu_h100_bf16_config \
119119
--dataset llm-finetune \
120120
dataset.dataset_name=gsm8k \
121121
checkpoint.pretrained_checkpoint=/checkpoints/base_model
@@ -129,7 +129,7 @@ For Energon/WebDataset data, create tar shards plus `.nv-meta` metadata and pass
129129

130130
```bash
131131
uv run python -m torch.distributed.run --nproc_per_node=1 scripts/training/run_recipe.py \
132-
--recipe qwen3_vl_8b_lora_1gpu_h100_bf16_energon_config \
132+
--recipe qwen3_vl_8b_peft_1gpu_h100_bf16_energon_config \
133133
--dataset vlm-energon \
134134
--step_func qwen3_vl_step \
135135
dataset.path=/data/vlm_energon \
@@ -144,7 +144,7 @@ For preloaded VLM JSON or JSONL, use records with `messages` or `conversations`
144144

145145
```bash
146146
uv run python -m torch.distributed.run --nproc_per_node=1 scripts/training/run_recipe.py \
147-
--recipe qwen3_vl_8b_lora_1gpu_h100_bf16_config \
147+
--recipe qwen3_vl_8b_peft_1gpu_h100_bf16_config \
148148
--dataset vlm-preloaded \
149149
--step_func qwen3_vl_step \
150150
dataset.train_data_path=/data/vlm/train.jsonl \

‎docs/recipe-usage.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ This guide will cover the next steps to make use of a training recipe, including
1212

1313
## Choosing a recipe or a new config
1414

15-
Start from an exported recipe when the model family and workflow already exist in `megatron.bridge.recipes`. Hardware-specific H100 recipes such as `llama3_8b_pretrain_2gpu_h100_bf16_config`, `llama32_1b_sft_1gpu_h100_bf16_config`, and `qwen3_8b_lora_1gpu_h100_bf16_config` are the canonical recipe names and share the same `<model>_<task>_<num>gpu_<gpu>_<dtype>_config` naming strategy as performance recipes. The older recipe names remain available as compatibility wrappers. Override those defaults for your dataset, checkpoint paths, run length, parallelism, or precision before creating a new recipe.
15+
Start from an exported recipe when the model family and workflow already exist in `megatron.bridge.recipes`. Hardware-specific H100 recipes such as `llama3_8b_pretrain_2gpu_h100_bf16_config`, `llama32_1b_sft_1gpu_h100_bf16_config`, and `qwen3_8b_peft_1gpu_h100_bf16_config` are the canonical recipe names and share the same `<model>_<task>_<num>gpu_<gpu>_<dtype>_config` naming strategy as performance recipes. The older recipe names remain available as compatibility wrappers. Override those defaults for your dataset, checkpoint paths, run length, parallelism, or precision before creating a new recipe.
1616

1717
Create a new recipe or config when the base model architecture is not represented by an existing model provider, the checkpoint conversion needs a new bridge, the forward step or dataset provider is model-specific, or you need a reusable configuration that will be shared across jobs. If the Hugging Face model is already supported by `AutoBridge`, you usually only need to start from the closest recipe and override the model provider or `hf_path`.
1818

‎docs/training/data-preparation.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -116,7 +116,7 @@ The generic launcher provides preset Hugging Face text datasets through `--datas
116116

117117
```bash
118118
uv run python -m torch.distributed.run --nproc_per_node=1 scripts/training/run_recipe.py \
119-
--recipe llama32_1b_lora_1gpu_h100_bf16_config \
119+
--recipe llama32_1b_peft_1gpu_h100_bf16_config \
120120
--dataset llm-finetune \
121121
dataset.dataset_name=gsm8k \
122122
checkpoint.pretrained_checkpoint=/checkpoints/base_model
@@ -130,7 +130,7 @@ For Energon/WebDataset data, create tar shards plus `.nv-meta` metadata and pass
130130

131131
```bash
132132
uv run python -m torch.distributed.run --nproc_per_node=1 scripts/training/run_recipe.py \
133-
--recipe qwen3_vl_8b_lora_1gpu_h100_bf16_energon_config \
133+
--recipe qwen3_vl_8b_peft_1gpu_h100_bf16_energon_config \
134134
--dataset vlm-energon \
135135
--step_func qwen3_vl_step \
136136
dataset.path=/data/vlm_energon \
@@ -145,7 +145,7 @@ For preloaded VLM JSON or JSONL, use records with `messages` or `conversations`
145145

146146
```bash
147147
uv run python -m torch.distributed.run --nproc_per_node=1 scripts/training/run_recipe.py \
148-
--recipe qwen3_vl_8b_lora_1gpu_h100_bf16_config \
148+
--recipe qwen3_vl_8b_peft_1gpu_h100_bf16_config \
149149
--dataset vlm-preloaded \
150150
--step_func qwen3_vl_step \
151151
dataset.train_data_path=/data/vlm/train.jsonl \

‎src/megatron/bridge/recipes/deepseek/deepseek_v2.py‎

Lines changed: 6 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -17,33 +17,22 @@
1717
from __future__ import annotations
1818

1919
from megatron.bridge.recipes.deepseek.h100 import deepseek_v2 as _h100_module
20-
from megatron.bridge.recipes.utils.aliases import call_h100_recipe
2120
from megatron.bridge.training.config import ConfigContainer
2221

2322

2423
AutoBridge = _h100_module.AutoBridge
2524

2625

27-
def deepseek_v2_lite_pretrain_config(*args: object, **kwargs: object) -> ConfigContainer:
26+
def deepseek_v2_lite_pretrain_config() -> ConfigContainer:
2827
"""Compatibility alias for ``deepseek_v2_lite_pretrain_8gpu_h100_bf16_config``."""
29-
return call_h100_recipe(
30-
globals(),
31-
_h100_module,
32-
_h100_module.deepseek_v2_lite_pretrain_8gpu_h100_bf16_config,
33-
*args,
34-
**kwargs,
35-
)
28+
_h100_module.AutoBridge = AutoBridge
29+
return _h100_module.deepseek_v2_lite_pretrain_8gpu_h100_bf16_config()
3630

3731

38-
def deepseek_v2_pretrain_config(*args: object, **kwargs: object) -> ConfigContainer:
32+
def deepseek_v2_pretrain_config() -> ConfigContainer:
3933
"""Compatibility alias for ``deepseek_v2_pretrain_128gpu_h100_bf16_config``."""
40-
return call_h100_recipe(
41-
globals(),
42-
_h100_module,
43-
_h100_module.deepseek_v2_pretrain_128gpu_h100_bf16_config,
44-
*args,
45-
**kwargs,
46-
)
34+
_h100_module.AutoBridge = AutoBridge
35+
return _h100_module.deepseek_v2_pretrain_128gpu_h100_bf16_config()
4736

4837

4938
__all__ = [

‎src/megatron/bridge/recipes/deepseek/deepseek_v3.py‎

Lines changed: 6 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -21,33 +21,22 @@
2121
_build_standalone_mtp_layout,
2222
set_deepseek_v3_pipeline_model_parallel_layout,
2323
)
24-
from megatron.bridge.recipes.utils.aliases import call_h100_recipe
2524
from megatron.bridge.training.config import ConfigContainer
2625

2726

2827
AutoBridge = _h100_module.AutoBridge
2928

3029

31-
def deepseek_v3_pretrain_config(*args: object, **kwargs: object) -> ConfigContainer:
30+
def deepseek_v3_pretrain_config() -> ConfigContainer:
3231
"""Compatibility alias for ``deepseek_v3_pretrain_1024gpu_h100_bf16_config``."""
33-
return call_h100_recipe(
34-
globals(),
35-
_h100_module,
36-
_h100_module.deepseek_v3_pretrain_1024gpu_h100_bf16_config,
37-
*args,
38-
**kwargs,
39-
)
32+
_h100_module.AutoBridge = AutoBridge
33+
return _h100_module.deepseek_v3_pretrain_1024gpu_h100_bf16_config()
4034

4135

42-
def deepseek_v3_pretrain_config_32nodes(*args: object, **kwargs: object) -> ConfigContainer:
36+
def deepseek_v3_pretrain_config_32nodes() -> ConfigContainer:
4337
"""Compatibility alias for ``deepseek_v3_pretrain_256gpu_h100_bf16_32nodes_config``."""
44-
return call_h100_recipe(
45-
globals(),
46-
_h100_module,
47-
_h100_module.deepseek_v3_pretrain_256gpu_h100_bf16_32nodes_config,
48-
*args,
49-
**kwargs,
50-
)
38+
_h100_module.AutoBridge = AutoBridge
39+
return _h100_module.deepseek_v3_pretrain_256gpu_h100_bf16_32nodes_config()
5140

5241

5342
__all__ = [

‎src/megatron/bridge/recipes/deepseek/deepseek_v4.py‎

Lines changed: 15 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -21,66 +21,40 @@
2121
DEEPSEEK_V4_FLASH_HF_PATH,
2222
_deepseek_v4_mxfp8_quant_recipe,
2323
)
24-
from megatron.bridge.recipes.utils.aliases import call_h100_recipe
2524
from megatron.bridge.training.config import ConfigContainer
2625

2726

2827
AutoBridge = _h100_module.AutoBridge
2928

3029

31-
def deepseek_v4_flash_no_mtp_sft_config(*args: object, **kwargs: object) -> ConfigContainer:
30+
def deepseek_v4_flash_no_mtp_sft_config() -> ConfigContainer:
3231
"""Compatibility alias for ``deepseek_v4_flash_no_mtp_sft_32gpu_h100_bf16_config``."""
33-
return call_h100_recipe(
34-
globals(),
35-
_h100_module,
36-
_h100_module.deepseek_v4_flash_no_mtp_sft_32gpu_h100_bf16_config,
37-
*args,
38-
**kwargs,
39-
)
32+
_h100_module.AutoBridge = AutoBridge
33+
return _h100_module.deepseek_v4_flash_no_mtp_sft_32gpu_h100_bf16_config()
4034

4135

42-
def deepseek_v4_flash_pretrain_config(*args: object, **kwargs: object) -> ConfigContainer:
36+
def deepseek_v4_flash_pretrain_config() -> ConfigContainer:
4337
"""Compatibility alias for ``deepseek_v4_flash_pretrain_32gpu_h100_bf16_config``."""
44-
return call_h100_recipe(
45-
globals(),
46-
_h100_module,
47-
_h100_module.deepseek_v4_flash_pretrain_32gpu_h100_bf16_config,
48-
*args,
49-
**kwargs,
50-
)
38+
_h100_module.AutoBridge = AutoBridge
39+
return _h100_module.deepseek_v4_flash_pretrain_32gpu_h100_bf16_config()
5140

5241

53-
def deepseek_v4_flash_pretrain_muon_config(*args: object, **kwargs: object) -> ConfigContainer:
42+
def deepseek_v4_flash_pretrain_muon_config() -> ConfigContainer:
5443
"""Compatibility alias for ``deepseek_v4_flash_pretrain_32gpu_h100_bf16_muon_config``."""
55-
return call_h100_recipe(
56-
globals(),
57-
_h100_module,
58-
_h100_module.deepseek_v4_flash_pretrain_32gpu_h100_bf16_muon_config,
59-
*args,
60-
**kwargs,
61-
)
44+
_h100_module.AutoBridge = AutoBridge
45+
return _h100_module.deepseek_v4_flash_pretrain_32gpu_h100_bf16_muon_config()
6246

6347

64-
def deepseek_v4_flash_pretrain_mxfp8_config(*args: object, **kwargs: object) -> ConfigContainer:
48+
def deepseek_v4_flash_pretrain_mxfp8_config() -> ConfigContainer:
6549
"""Compatibility alias for ``deepseek_v4_flash_pretrain_32gpu_h100_fp8mx_config``."""
66-
return call_h100_recipe(
67-
globals(),
68-
_h100_module,
69-
_h100_module.deepseek_v4_flash_pretrain_32gpu_h100_fp8mx_config,
70-
*args,
71-
**kwargs,
72-
)
50+
_h100_module.AutoBridge = AutoBridge
51+
return _h100_module.deepseek_v4_flash_pretrain_32gpu_h100_fp8mx_config()
7352

7453

75-
def deepseek_v4_flash_sft_config(*args: object, **kwargs: object) -> ConfigContainer:
54+
def deepseek_v4_flash_sft_config() -> ConfigContainer:
7655
"""Compatibility alias for ``deepseek_v4_flash_sft_32gpu_h100_bf16_config``."""
77-
return call_h100_recipe(
78-
globals(),
79-
_h100_module,
80-
_h100_module.deepseek_v4_flash_sft_32gpu_h100_bf16_config,
81-
*args,
82-
**kwargs,
83-
)
56+
_h100_module.AutoBridge = AutoBridge
57+
return _h100_module.deepseek_v4_flash_sft_32gpu_h100_bf16_config()
8458

8559

8660
__all__ = [

‎src/megatron/bridge/recipes/gemma/gemma2.py‎

Lines changed: 39 additions & 84 deletions
Original file line numberDiff line numberDiff line change
@@ -16,114 +16,69 @@
1616

1717
from __future__ import annotations
1818

19+
from megatron.bridge.peft.base import PEFT
1920
from megatron.bridge.recipes.gemma.h100 import gemma2 as _h100_module
2021
from megatron.bridge.recipes.gemma.h100.gemma2 import (
2122
_adjust_gemma2_vocab_size,
2223
)
23-
from megatron.bridge.recipes.utils.aliases import call_h100_recipe
2424
from megatron.bridge.training.config import ConfigContainer
2525

2626

2727
AutoBridge = _h100_module.AutoBridge
2828

2929

30-
def gemma2_27b_peft_config(*args: object, **kwargs: object) -> ConfigContainer:
31-
"""Compatibility alias for ``gemma2_27b_lora_4gpu_h100_bf16_config``."""
32-
return call_h100_recipe(
33-
globals(),
34-
_h100_module,
35-
_h100_module.gemma2_27b_lora_4gpu_h100_bf16_config,
36-
*args,
37-
**kwargs,
38-
)
30+
def gemma2_27b_peft_config(peft_scheme: str | PEFT = "lora") -> ConfigContainer:
31+
"""Compatibility alias for ``gemma2_27b_peft_4gpu_h100_bf16_config``."""
32+
_h100_module.AutoBridge = AutoBridge
33+
return _h100_module.gemma2_27b_peft_4gpu_h100_bf16_config(peft_scheme=peft_scheme)
3934

4035

41-
def gemma2_27b_pretrain_config(*args: object, **kwargs: object) -> ConfigContainer:
36+
def gemma2_27b_pretrain_config() -> ConfigContainer:
4237
"""Compatibility alias for ``gemma2_27b_pretrain_16gpu_h100_bf16_config``."""
43-
return call_h100_recipe(
44-
globals(),
45-
_h100_module,
46-
_h100_module.gemma2_27b_pretrain_16gpu_h100_bf16_config,
47-
*args,
48-
**kwargs,
49-
)
38+
_h100_module.AutoBridge = AutoBridge
39+
return _h100_module.gemma2_27b_pretrain_16gpu_h100_bf16_config()
5040

5141

52-
def gemma2_27b_sft_config(*args: object, **kwargs: object) -> ConfigContainer:
42+
def gemma2_27b_sft_config() -> ConfigContainer:
5343
"""Compatibility alias for ``gemma2_27b_sft_16gpu_h100_bf16_config``."""
54-
return call_h100_recipe(
55-
globals(),
56-
_h100_module,
57-
_h100_module.gemma2_27b_sft_16gpu_h100_bf16_config,
58-
*args,
59-
**kwargs,
60-
)
61-
62-
63-
def gemma2_2b_peft_config(*args: object, **kwargs: object) -> ConfigContainer:
64-
"""Compatibility alias for ``gemma2_2b_lora_1gpu_h100_bf16_config``."""
65-
return call_h100_recipe(
66-
globals(),
67-
_h100_module,
68-
_h100_module.gemma2_2b_lora_1gpu_h100_bf16_config,
69-
*args,
70-
**kwargs,
71-
)
72-
73-
74-
def gemma2_2b_pretrain_config(*args: object, **kwargs: object) -> ConfigContainer:
44+
_h100_module.AutoBridge = AutoBridge
45+
return _h100_module.gemma2_27b_sft_16gpu_h100_bf16_config()
46+
47+
48+
def gemma2_2b_peft_config(peft_scheme: str | PEFT = "lora") -> ConfigContainer:
49+
"""Compatibility alias for ``gemma2_2b_peft_1gpu_h100_bf16_config``."""
50+
_h100_module.AutoBridge = AutoBridge
51+
return _h100_module.gemma2_2b_peft_1gpu_h100_bf16_config(peft_scheme=peft_scheme)
52+
53+
54+
def gemma2_2b_pretrain_config() -> ConfigContainer:
7555
"""Compatibility alias for ``gemma2_2b_pretrain_2gpu_h100_bf16_config``."""
76-
return call_h100_recipe(
77-
globals(),
78-
_h100_module,
79-
_h100_module.gemma2_2b_pretrain_2gpu_h100_bf16_config,
80-
*args,
81-
**kwargs,
82-
)
56+
_h100_module.AutoBridge = AutoBridge
57+
return _h100_module.gemma2_2b_pretrain_2gpu_h100_bf16_config()
8358

8459

85-
def gemma2_2b_sft_config(*args: object, **kwargs: object) -> ConfigContainer:
60+
def gemma2_2b_sft_config() -> ConfigContainer:
8661
"""Compatibility alias for ``gemma2_2b_sft_1gpu_h100_bf16_config``."""
87-
return call_h100_recipe(
88-
globals(),
89-
_h100_module,
90-
_h100_module.gemma2_2b_sft_1gpu_h100_bf16_config,
91-
*args,
92-
**kwargs,
93-
)
94-
95-
96-
def gemma2_9b_peft_config(*args: object, **kwargs: object) -> ConfigContainer:
97-
"""Compatibility alias for ``gemma2_9b_lora_1gpu_h100_bf16_config``."""
98-
return call_h100_recipe(
99-
globals(),
100-
_h100_module,
101-
_h100_module.gemma2_9b_lora_1gpu_h100_bf16_config,
102-
*args,
103-
**kwargs,
104-
)
105-
106-
107-
def gemma2_9b_pretrain_config(*args: object, **kwargs: object) -> ConfigContainer:
62+
_h100_module.AutoBridge = AutoBridge
63+
return _h100_module.gemma2_2b_sft_1gpu_h100_bf16_config()
64+
65+
66+
def gemma2_9b_peft_config(peft_scheme: str | PEFT = "lora") -> ConfigContainer:
67+
"""Compatibility alias for ``gemma2_9b_peft_1gpu_h100_bf16_config``."""
68+
_h100_module.AutoBridge = AutoBridge
69+
return _h100_module.gemma2_9b_peft_1gpu_h100_bf16_config(peft_scheme=peft_scheme)
70+
71+
72+
def gemma2_9b_pretrain_config() -> ConfigContainer:
10873
"""Compatibility alias for ``gemma2_9b_pretrain_8gpu_h100_bf16_config``."""
109-
return call_h100_recipe(
110-
globals(),
111-
_h100_module,
112-
_h100_module.gemma2_9b_pretrain_8gpu_h100_bf16_config,
113-
*args,
114-
**kwargs,
115-
)
74+
_h100_module.AutoBridge = AutoBridge
75+
return _h100_module.gemma2_9b_pretrain_8gpu_h100_bf16_config()
11676

11777

118-
def gemma2_9b_sft_config(*args: object, **kwargs: object) -> ConfigContainer:
78+
def gemma2_9b_sft_config() -> ConfigContainer:
11979
"""Compatibility alias for ``gemma2_9b_sft_4gpu_h100_bf16_config``."""
120-
return call_h100_recipe(
121-
globals(),
122-
_h100_module,
123-
_h100_module.gemma2_9b_sft_4gpu_h100_bf16_config,
124-
*args,
125-
**kwargs,
126-
)
80+
_h100_module.AutoBridge = AutoBridge
81+
return _h100_module.gemma2_9b_sft_4gpu_h100_bf16_config()
12782

12883

12984
__all__ = [

0 commit comments

Comments
 (0)