Skip to content

Commit 0dae728

Browse files
authored
[examples] refactor: Organize model examples by brand (#3847)
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
1 parent 544955c commit 0dae728

136 files changed

Lines changed: 152 additions & 137 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎README.md‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -11,33 +11,33 @@
1111
</div>
1212

1313
## 📣 News
14-
- [05/06/2026] [**Gemma 4 VL 26B-A4B**](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/models/vlm/gemma4_vl) is now supported! Checkpoint conversion, SFT, and PEFT (LoRA) recipes for Google's MoE vision-language model (26B total / 4B active params, 128 experts top-k=8, dual sliding/global attention with K=V tying on full-attention layers) are available on **main**. See the [examples README](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/models/vlm/gemma4_vl/README.md) for the full walkthrough.
14+
- [05/06/2026] [**Gemma 4 VL 26B-A4B**](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/models/gemma/gemma4_vl) is now supported! Checkpoint conversion, SFT, and PEFT (LoRA) recipes for Google's MoE vision-language model (26B total / 4B active params, 128 experts top-k=8, dual sliding/global attention with K=V tying on full-attention layers) are available on **main**. See the [examples README](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/models/gemma/gemma4_vl/README.md) for the full walkthrough.
1515

1616
- [04/28/2026] Day 0 support for [**Nemotron-3 Nano Omni**](https://huggingface.co/nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16), a 30B-A3B MoE multimodal model that jointly processes image, video, audio, and text. Checkpoint conversion, SFT, and LoRA recipes are available on the [`nemotron_3_omni`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/nemotron_3_omni) branch — see the [examples README](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/nemotron_3_omni/examples/models/vlm/nemotron_3_omni/README.md) for the full walkthrough.
1717

1818
- [04/19/2026] [**Qwen3.6-35B-A3B**](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) is now supported! Qwen3.6 uses the same architecture as Qwen3.5 VL MoE (`Qwen3_5MoeForConditionalGeneration`) and works with the existing [Qwen3.5-VL bridge](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/src/megatron/bridge/models/qwen_vl) out of the box — no code changes needed. HF→Megatron conversion and inference verified.
1919

2020
- [04/16/2026] **Megatron Bridge 0.4.0 released!** New model support (Kimi 2.5, Nemotron 3 Super, Qwen 3.5 VL, MiniMax M2, Sarvam, MiMo, and more), diffusion model collection, sequence-packing improvements, FP8 export, pruning & quantization, Transformers 5.x compatibility, and Python 3.12 migration. Huge thanks to our community contributors: [@HollowMan6](https://github.com/HollowMan6), [@shaltielshmid](https://github.com/shaltielshmid), [@jaeminh](https://github.com/jaeminh), [@pavelgein](https://github.com/pavelgein), [@ShiftyBlock](https://github.com/ShiftyBlock), [@erictang000](https://github.com/erictang000), [@eternally-z](https://github.com/eternally-z), [@Hayak3](https://github.com/Hayak3), and [@mohit-sarvam](https://github.com/mohit-sarvam)! See the [full release notes](https://github.com/NVIDIA-NeMo/Megatron-Bridge/releases/tag/v0.4.0).
2121

22-
- [04/12/2026] [**MiniMax-M2.5 / M2.7**](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/models/minimax_m2) are now supported! Both models share the same architecture as MiniMax-M2 and work with the existing bridge out of the box — checkpoint conversion and inference verified on real FP8 checkpoints.
22+
- [04/12/2026] [**MiniMax-M2.5 / M2.7**](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/models/minimax/minimax_m2) are now supported! Both models share the same architecture as MiniMax-M2 and work with the existing bridge out of the box — checkpoint conversion and inference verified on real FP8 checkpoints.
2323

24-
- [04/10/2026] [**Qwen3-ASR**](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/models/audio_lm/qwen3_asr) is now supported! Checkpoint conversion and inference for [Qwen3's ASR model](https://github.com/QwenLM/Qwen3-ASR) are available on **main**.
24+
- [04/10/2026] [**Qwen3-ASR**](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/models/qwen/qwen3_asr) is now supported! Checkpoint conversion and inference for [Qwen3's ASR model](https://github.com/QwenLM/Qwen3-ASR) are available on **main**.
2525

2626
- [04/09/2026] [**Bailing MoE V2**](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/models/bailing) is now supported! Checkpoint conversion and inference for the Bailing MoE V2 model are available on **main**. Thank you to [@ccclyu](https://github.com/ccclyu) for the community contribution!
2727

2828
- [04/07/2026] Megatron Bridge’s PEFT support was featured at [PyTorch Conference Europe 2026 Talk](https://pytorchconferenceeu2026.sched.com/event/2Juce/optimizing-reinforcement-learning-at-trillion-parameter-scale-songlin-jiang-aalto-university-mind-lab).
2929

30-
- [04/01/2026] [**Kimi K2.5 VL**](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/models/vlm/kimi_k25_vl) is now supported! Checkpoint conversion, inference, and training recipes for [Moonshot AI’s Kimi-K2.5-VL](https://huggingface.co/moonshotai/Kimi-K2.5) vision-language model are available on **main**.
30+
- [04/01/2026] [**Kimi K2.5 VL**](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/models/kimi/kimi_k25_vl) is now supported! Checkpoint conversion, inference, and training recipes for [Moonshot AI’s Kimi-K2.5-VL](https://huggingface.co/moonshotai/Kimi-K2.5) vision-language model are available on **main**.
3131

3232
- [03/31/2026] **Agent Skills for Megatron Bridge!** We've added a [`skills/`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/skills) directory with structured guides that AI coding agents (Cursor, Claude Code, Codex, etc.) can use to help you add model support, set up dev environments, tune performance, and more. Try them out, and PRs to improve or add new skills are very welcome!
3333

34-
- [03/26/2026] [**Nemotron 3 Super**](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/models/nemotron_3) is now on **main**! Checkpoint conversion and SFT/LoRA recipes (120B-A12B) are available in the main branch. Read the [blog post](https://developer.nvidia.com/blog/introducing-nemotron-3-super-an-open-hybrid-mamba-transformer-moe-for-agentic-reasoning/).
34+
- [03/26/2026] [**Nemotron 3 Super**](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/models/nemotron/nemotron_3) is now on **main**! Checkpoint conversion and SFT/LoRA recipes (120B-A12B) are available in the main branch. Read the [blog post](https://developer.nvidia.com/blog/introducing-nemotron-3-super-an-open-hybrid-mamba-transformer-moe-for-agentic-reasoning/).
3535

3636
- [03/12/2026] **Deprecating Python 3.10 support:** We're officially dropping Python 3.10 support with the upcoming 0.4.0 release. Downstream applications must raise their lower boundary to 3.12 to stay compatible with Megatron-Bridge.
3737

3838
- [12/16/2025] [Mind Lab](https://macaron.im/mindlab) successfully used Megatron-bridge and [VeRL](https://github.com/volcengine/verl) to trained GRPO Lora for Trillion-parameter model on 64 H800 - See their [techblog](https://macaron.im/mindlab/research/building-trillion-parameter-reasoning-rl-with-10-gpus).
3939

40-
- [12/15/2025] Day 0 support for [NVIDIA-NeMotron-3-Nano-30B-A3B-FP8](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8)! [Reproducible code](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/models/nemotron_3/nano) and custom NGC container: [nvcr.io/nvidia/nemo:25.11.nemotron_3_nano](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/nemo?version=25.11.nemotron_3_nano)
40+
- [12/15/2025] Day 0 support for [NVIDIA-NeMotron-3-Nano-30B-A3B-FP8](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8)! [Reproducible code](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/models/nemotron/nemotron_3/nano) and custom NGC container: [nvcr.io/nvidia/nemo:25.11.nemotron_3_nano](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/nemo?version=25.11.nemotron_3_nano)
4141

4242
## Overview
4343

‎docs/models/llm/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ Megatron Bridge supports the following LLM families:
1919
| **LLaMA Nemotron** | [llama-nemotron.md](llama-nemotron.md) | NVIDIA LLaMA Nemotron models |
2020
| **Mistral** | [mistral.md](mistral.md) | Mistral AI models |
2121
| **Moonlight** | [moonlight.md](moonlight.md) | Moonlight model family |
22-
| **Nemotron-3** | [nemotron3.md](nemotron3.md) | NVIDIA Nemotron-3 models |
22+
| **Nemotron-3 Nano** | [nemotron3-nano.md](nemotron3-nano.md) | NVIDIA Nemotron-3 Nano models |
2323
| **Nemotron-3 Super** | [nemotron3-super.md](nemotron3-super.md) | NVIDIA Nemotron-3 Super models |
2424
| **Nemotron-H** | [nemotronh.md](nemotronh.md) | NVIDIA Nemotron-H models |
2525
| **OLMoE** | [olmoe.md](olmoe.md) | OLMoE (Open Language Model - Mixture of Experts) |

‎docs/models/llm/gemma3.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -182,7 +182,7 @@ uv run python -m torch.distributed.run --nproc-per-node=8 run/run_recipe.py \
182182
- Gemma 3 1B: https://huggingface.co/google/gemma-3-1b-it
183183

184184
## Related Docs
185-
- Gemma3 Vision-Language Models: [Gemma 3 VL](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/models/vlm/gemma3_vl/README.md)
185+
- Gemma3 Vision-Language Models: [Gemma 3 VL](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/models/gemma/gemma3_vl/README.md)
186186
- Recipe usage: [Recipe usage](../../recipe-usage.md)
187187
- Customizing the training recipe configuration: [Configuration overview](../../training/config-container-overview.md)
188188
- Training entry points: [Entry points](../../training/entry-points.md)

‎docs/models/llm/index.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,9 +15,9 @@ llama3.md
1515
llama-nemotron.md
1616
mistral.md
1717
moonlight.md
18-
nemotron3.md
18+
nemotron3-nano.md
1919
nemotron3-super.md
2020
nemotronh.md
2121
olmoe.md
2222
qwen.md
23-
```
23+
```
Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ uv run python examples/conversion/convert_checkpoints.py export \
3838
BLEND_PATH=/path/to/dataset/blend
3939
TOKENIZER_MODEL=/path/to/tiktok/tokenizer/model
4040

41-
uv run python -m torch.distributed.run --nproc-per-node=8 examples/models/nemotron_3/pretrain_nemotron_3_nano.py \
41+
uv run python -m torch.distributed.run --nproc-per-node=8 examples/models/nemotron/nemotron_3/nano/pretrain_nemotron_3_nano.py \
4242
--per-split-data-args-path=${BLEND_PATH} \
4343
--tokenizer-model=${TOKENIZER_MODEL} \
4444
train.global_batch_size=3072 \
@@ -56,7 +56,7 @@ Notes:
5656

5757
### Full Parameter Fine-Tuning
5858
```bash
59-
uv run python -m torch.distributed.run --nproc-per-node=8 examples/models/nemotron_3/finetune_nemotron_3_nano.py \
59+
uv run python -m torch.distributed.run --nproc-per-node=8 examples/models/nemotron/nemotron_3/nano/finetune_nemotron_3_nano.py \
6060
train.global_batch_size=128 \
6161
train.train_iters=100 \
6262
scheduler.lr_warmup_iters=10 \
@@ -72,7 +72,7 @@ Notes:
7272
### LoRA Fine-Tuning
7373
To enable LoRA fine-tuning, pass `--peft lora` to script
7474
```bash
75-
uv run python -m torch.distributed.run --nproc-per-node=8 examples/models/nemotron_3/finetune_nemotron_3_nano.py \
75+
uv run python -m torch.distributed.run --nproc-per-node=8 examples/models/nemotron/nemotron_3/nano/finetune_nemotron_3_nano.py \
7676
--peft lora \
7777
train.global_batch_size=128 \
7878
train.train_iters=100 \

‎docs/models/llm/nemotron3-super.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -120,7 +120,7 @@ uv run python -m torch.distributed.run --nproc-per-node=8 examples/conversion/co
120120
BLEND_PATH=/path/to/dataset/blend.json
121121
CHECKPOINT_DIR=/path/to/checkpoints
122122

123-
uv run python -m torch.distributed.run --nproc-per-node=8 examples/models/nemotron_3/pretrain_nemotron_3_super.py \
123+
uv run python -m torch.distributed.run --nproc-per-node=8 examples/models/nemotron/nemotron_3/super/pretrain_nemotron_3_super.py \
124124
--per-split-data-args-path=${BLEND_PATH} \
125125
logger.wandb_project=your_project \
126126
logger.wandb_entity=nvidia \
@@ -153,7 +153,7 @@ For quick testing without a dataset:
153153
```bash
154154
CHECKPOINT_DIR=/path/to/checkpoints
155155

156-
uv run python -m torch.distributed.run --nproc-per-node=8 examples/models/nemotron_3/pretrain_nemotron_3_super.py \
156+
uv run python -m torch.distributed.run --nproc-per-node=8 examples/models/nemotron/nemotron_3/super/pretrain_nemotron_3_super.py \
157157
logger.wandb_project=your_project \
158158
logger.wandb_entity=nvidia \
159159
checkpoint.load=${CHECKPOINT_DIR} \
@@ -179,7 +179,7 @@ Notes:
179179
MEGATRON_PATH=/path/to/pretrained/megatron/ckpt
180180
CHECKPOINT_DIR=/path/to/finetuned/checkpoints
181181

182-
uv run python -m torch.distributed.run --nproc-per-node=8 examples/models/nemotron_3/finetune_nemotron_3_super.py \
182+
uv run python -m torch.distributed.run --nproc-per-node=8 examples/models/nemotron/nemotron_3/super/finetune_nemotron_3_super.py \
183183
logger.wandb_project=your_project \
184184
logger.wandb_entity=nvidia \
185185
logger.log_interval=5 \
@@ -208,7 +208,7 @@ To enable LoRA fine-tuning, pass `--peft lora` to the script:
208208
MEGATRON_PATH=/path/to/pretrained/megatron/ckpt
209209
CHECKPOINT_DIR=/path/to/lora/checkpoints
210210

211-
uv run python -m torch.distributed.run --nproc-per-node=8 examples/models/nemotron_3/finetune_nemotron_3_super.py \
211+
uv run python -m torch.distributed.run --nproc-per-node=8 examples/models/nemotron/nemotron_3/super/finetune_nemotron_3_super.py \
212212
--peft lora \
213213
logger.wandb_project=your_project \
214214
logger.wandb_entity=nvidia \
@@ -303,7 +303,7 @@ After quantization, further improve model quality with QAT by continuing trainin
303303
MEGATRON_PATH=/path/to/quantized/megatron/ckpt
304304
CHECKPOINT_DIR=/path/to/qat/checkpoints
305305

306-
uv run python -m torch.distributed.run --nproc-per-node=8 examples/models/nemotron_3/qat_nemotron_3_super.py \
306+
uv run python -m torch.distributed.run --nproc-per-node=8 examples/models/nemotron/nemotron_3/super/qat_nemotron_3_super.py \
307307
--megatron-load-path=${MEGATRON_PATH} \
308308
--seq-length=8192 \
309309
--packed-sequence \

‎docs/models/vlm/gemma3-vl.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ Gemma 3 VL builds on the Gemma 3 architecture with additional multimodal capabil
4646

4747
## Examples
4848

49-
For checkpoint conversion, inference, finetuning recipes, and step-by-step training guides, see the [Gemma 3 VL Examples](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/models/vlm/gemma3_vl/README.md).
49+
For checkpoint conversion, inference, finetuning recipes, and step-by-step training guides, see the [Gemma 3 VL Examples](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/models/gemma/gemma3_vl/README.md).
5050

5151
## Hugging Face Model Cards
5252

‎docs/models/vlm/gemma4-vl.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,4 +12,4 @@ PEFT on image-text datasets. The finetuned model can be re-exported to
1212

1313
For the full setup, conversion, inference, training, and LoRA merge /
1414
adapter export workflows, see
15-
[`examples/models/vlm/gemma4_vl/README.md`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/models/vlm/gemma4_vl/README.md).
15+
[`examples/models/gemma/gemma4_vl/README.md`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/models/gemma/gemma4_vl/README.md).

‎docs/models/vlm/glm-45v.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ GLM-4.5V combines efficient sparse MoE language modeling with multimodal capabil
4141

4242
## Examples
4343

44-
For checkpoint conversion, inference, finetuning recipes, and step-by-step training guides, see the [GLM-4.5V Examples](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/models/vlm/glm_45v/README.md).
44+
For checkpoint conversion, inference, finetuning recipes, and step-by-step training guides, see the [GLM-4.5V Examples](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/models/glm/glm_45v/README.md).
4545

4646
## Hugging Face Model Cards
4747

‎docs/models/vlm/ministral3.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ Ministral 3 combines efficient language modeling with multimodal capabilities:
5151

5252
## Examples
5353

54-
For checkpoint conversion, inference, finetuning recipes, and step-by-step training guides, see the [Ministral 3 Examples](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/models/vlm/ministral3/README.md).
54+
For checkpoint conversion, inference, finetuning recipes, and step-by-step training guides, see the [Ministral 3 Examples](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/models/ministral/ministral3/README.md).
5555

5656
## Hugging Face Model Cards
5757

0 commit comments

Comments
 (0)