From 7b5300bc4ad6ccf165894bd7d20ebe5dbc6e79a3 Mon Sep 17 00:00:00 2001 From: Andrew Chen Date: Mon, 8 Jun 2026 11:48:07 +0000 Subject: [PATCH 1/5] =?UTF-8?q?docs:=20Sphinx/MyST=20->=20Fern=20MDX=20mig?= =?UTF-8?q?ration=20=E2=80=94=20WIP=20checkpoint?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit De-risk commit: capturing the stranded working-tree migration state (114 deletions of Sphinx/MyST .md, 56 modified, 124 .mdx authored, docs/fern/ infra present). Checkpoint only โ€” completeness + template-conformance pass not yet verified; not for merge as-is. Signed-off-by: Andrew Chen --- .../ISSUE_TEMPLATE/model_support_request.yml | 2 +- README.md | 34 +-- docs/README.md | 269 ----------------- .../architecture.mdx} | 31 +- docs/about/index.mdx | 59 ++++ .../release-notes/changelog.mdx} | 5 + docs/about/release-notes/index.mdx | 88 ++++++ .../release-notes/software-versions.mdx} | 21 +- docs/autodoc2_docstrings_parser.py | 35 --- docs/broken_links_false_positives.json | 4 - docs/conf.py | 144 ---------- docs/documentation.md | 145 ---------- docs/fern/docs.yml | 33 +++ docs/fern/fern.config.json | 4 + docs/fern/versions/nightly.yml | 50 ++++ docs/get-started/index.mdx | 46 +++ docs/get-started/installation.mdx | 55 ++++ docs/get-started/migration/index.mdx | 20 ++ .../megatron-lm-to-megatron-bridge.mdx} | 15 +- .../migration/nemo2-to-megatron-bridge.mdx} | 159 +++++----- docs/get-started/prerequisites.mdx | 32 +++ .../quickstart/hugging-face-conversion.mdx} | 23 +- docs/get-started/quickstart/index.mdx | 48 ++++ docs/index.md | 130 --------- docs/index.mdx | 38 +++ .../distillation.mdx} | 5 + .../index.mdx} | 13 +- .../pruning.mdx} | 7 +- .../quantization.mdx} | 7 +- docs/models/README.md | 96 ------- docs/models/bailing/{index.md => index.mdx} | 13 +- docs/models/bailing/{ling-2.md => ling-2.mdx} | 6 +- .../{deepseek-v2.md => deepseek-v2.mdx} | 18 +- .../{deepseek-v3.md => deepseek-v3.mdx} | 17 +- .../{deepseek-v4.md => deepseek-v4.mdx} | 7 +- docs/models/deepseek/index.md | 17 -- docs/models/deepseek/index.mdx | 14 + .../falcon/{falcon-h1.md => falcon-h1.mdx} | 6 +- docs/models/falcon/{index.md => index.mdx} | 13 +- docs/models/gemma/{gemma.md => gemma.mdx} | 6 +- docs/models/gemma/{gemma2.md => gemma2.mdx} | 22 +- .../gemma/{gemma3-vl.md => gemma3-vl.mdx} | 13 +- docs/models/gemma/{gemma3.md => gemma3.mdx} | 21 +- .../gemma/{gemma4-vl.md => gemma4-vl.mdx} | 7 +- docs/models/gemma/index.md | 21 -- docs/models/gemma/index.mdx | 16 ++ docs/models/glm/{glm-45v.md => glm-45v.mdx} | 17 +- docs/models/glm/{glm45.md => glm45.mdx} | 20 +- docs/models/glm/{glm47.md => glm47.mdx} | 6 +- docs/models/glm/{glm5.md => glm5.mdx} | 6 +- docs/models/glm/index.md | 19 -- docs/models/glm/index.mdx | 15 + .../gpt-oss.md => gpt-oss/gpt-oss.mdx} | 17 +- .../{gpt_oss/index.md => gpt-oss/index.mdx} | 13 +- docs/models/index.mdx | 101 +++++++ docs/models/kimi/index.md | 15 - docs/models/kimi/index.mdx | 13 + docs/models/kimi/{kimi-k2.md => kimi-k2.mdx} | 5 + .../kimi/{kimi-k25-vl.md => kimi-k25-vl.mdx} | 6 +- docs/models/llama/{index.md => index.mdx} | 16 +- docs/models/llama/{llama2.md => llama2.mdx} | 6 +- docs/models/llama/{llama3.md => llama3.mdx} | 17 +- docs/models/mimo/{index.md => index.mdx} | 13 +- docs/models/mimo/{mimo.md => mimo.mdx} | 6 +- docs/models/minimax/index.md | 13 - docs/models/minimax/index.mdx | 12 + .../minimax/{minimax-m2.md => minimax-m2.mdx} | 6 +- docs/models/mistral/index.md | 15 - docs/models/mistral/index.mdx | 13 + .../mistral/{ministral3.md => ministral3.mdx} | 17 +- .../mistral/{mistral.md => mistral.mdx} | 17 +- docs/models/moonlight/{index.md => index.mdx} | 13 +- .../moonlight/{moonlight.md => moonlight.mdx} | 18 +- docs/models/nemotron/index.md | 28 -- docs/models/nemotron/index.mdx | 22 ++ .../{llama-nemotron.md => llama-nemotron.mdx} | 16 +- ...nemotron-3-omni.md => nemotron-3-omni.mdx} | 11 +- ...-nano-v2-vl.md => nemotron-nano-v2-vl.mdx} | 17 +- .../{nemotron3-nano.md => nemotron3-nano.mdx} | 19 +- ...nemotron3-super.md => nemotron3-super.mdx} | 21 +- .../nemotron/{nemotronh.md => nemotronh.mdx} | 22 +- docs/models/olmoe/{index.md => index.mdx} | 13 +- docs/models/olmoe/{olmoe.md => olmoe.mdx} | 18 +- docs/models/qwen/index.md | 31 -- docs/models/qwen/index.mdx | 21 ++ docs/models/qwen/{qwen.md => qwen.mdx} | 27 +- .../qwen/{qwen2-audio.md => qwen2-audio.mdx} | 6 +- .../qwen/{qwen2.5-vl.md => qwen2.5-vl.mdx} | 14 +- .../qwen/{qwen25-omni.md => qwen25-omni.mdx} | 6 +- .../qwen/{qwen3-asr.md => qwen3-asr.mdx} | 6 +- .../qwen/{qwen3-moe.md => qwen3-moe.mdx} | 8 +- .../qwen/{qwen3-next.md => qwen3-next.mdx} | 8 +- .../qwen/{qwen3-omni.md => qwen3-omni.mdx} | 15 +- .../models/qwen/{qwen3-vl.md => qwen3-vl.mdx} | 11 +- .../qwen/{qwen35-vl.md => qwen35-vl.mdx} | 19 +- docs/models/sarvam/{index.md => index.mdx} | 13 +- docs/models/sarvam/{sarvam.md => sarvam.mdx} | 6 +- docs/models/stepfun/{index.md => index.mdx} | 14 +- docs/models/stepfun/{step35.md => step35.mdx} | 6 +- docs/project.json | 4 - docs/reference/api/index.mdx | 18 ++ docs/reference/index.mdx | 24 ++ .../performance/archive.mdx} | 7 +- docs/reference/performance/index.mdx | 24 ++ .../performance/summary.mdx} | 7 +- docs/releases/README.md | 83 ------ docs/resources/agent-skills.mdx | 11 + .../contributing/cherrypick-policy.mdx} | 11 +- docs/resources/contributing/index.mdx | 24 ++ .../contributing/release-process.mdx} | 7 +- docs/resources/documentation-development.mdx | 59 ++++ docs/resources/documentation-map.mdx | 272 ++++++++++++++++++ docs/resources/index.mdx | 28 ++ docs/skills-index.md | 69 ----- docs/training/README.md | 126 -------- ...tation.md => activation-recomputation.mdx} | 13 +- ...zations.md => attention-optimizations.mdx} | 5 + docs/training/{callbacks.md => callbacks.mdx} | 23 +- .../{checkpointing.md => checkpointing.mdx} | 25 +- ...n-overlap.md => communication-overlap.mdx} | 28 +- ...rview.md => config-container-overview.mdx} | 5 + .../{cpu-offloading.md => cpu-offloading.mdx} | 19 +- .../{cuda-graphs.md => cuda-graphs.mdx} | 15 +- docs/training/data-preparation.mdx | 168 +++++++++++ .../{entry-points.md => entry-points.mdx} | 13 +- ...l.md => hierarchical-context-parallel.mdx} | 15 +- docs/training/index.mdx | 129 +++++++++ docs/training/{logging.md => logging.mdx} | 38 +-- .../{megatron-fsdp.md => megatron-fsdp.mdx} | 17 +- ...mory-estimator.md => memory-estimator.mdx} | 11 +- ...mixed-precision.md => mixed-precision.mdx} | 19 +- ...e-optimization.md => moe-optimization.mdx} | 19 +- ...ediction.md => multi-token-prediction.mdx} | 19 +- ...r-scheduler.md => optimizer-scheduler.mdx} | 7 +- ...cked-sequences.md => packed-sequences.mdx} | 21 +- .../parallelisms.mdx} | 15 +- docs/training/{peft.md => peft.mdx} | 32 +-- .../performance-tuning.mdx} | 19 +- docs/training/{profiling.md => profiling.mdx} | 15 +- .../recipe-usage.mdx} | 11 +- .../{resiliency.md => resiliency.mdx} | 17 +- ...settings.md => training-loop-settings.mdx} | 10 +- docs/troubleshooting/index.mdx | 37 +++ .../known-issues.mdx} | 5 + docs/use-cases/index.mdx | 20 ++ .../model-development/add-new-model.mdx} | 40 +-- docs/use-cases/model-development/index.mdx | 20 ++ .../reinforcement-learning/index.mdx | 20 ++ .../rl-framework-integration.mdx} | 17 +- docs/versions1.json | 37 --- examples/models/deepseek_v4/README.md | 2 +- examples/models/gemma/gemma3_vl/README.md | 4 +- examples/models/glm/glm_45v/README.md | 4 +- examples/models/mistral/ministral3/README.md | 4 +- examples/models/qwen/qwen35_vl/README.md | 2 +- examples/models/qwen/qwen3_omni/README.md | 2 +- examples/models/qwen/qwen3_vl/README.md | 2 +- .../nemo-mbridge-mlm-bridge-training/SKILL.md | 4 +- .../card.yaml | 4 +- .../skill-card.md | 2 +- .../SKILL.md | 2 +- .../card.yaml | 2 +- .../skill-card.md | 2 +- .../nemo-mbridge-perf-cpu-offloading/SKILL.md | 2 +- .../card.yaml | 2 +- .../skill-card.md | 2 +- skills/nemo-mbridge-perf-cuda-graphs/SKILL.md | 6 +- .../nemo-mbridge-perf-cuda-graphs/card.yaml | 6 +- .../skill-card.md | 4 +- .../SKILL.md | 2 +- .../card.yaml | 2 +- .../skill-card.md | 4 +- .../SKILL.md | 2 +- .../card.yaml | 2 +- .../skill-card.md | 2 +- .../nemo-mbridge-perf-megatron-fsdp/SKILL.md | 2 +- .../skill-card.md | 2 +- .../nemo-mbridge-perf-memory-tuning/SKILL.md | 6 +- .../nemo-mbridge-perf-memory-tuning/card.yaml | 4 +- .../skill-card.md | 4 +- .../SKILL.md | 2 +- .../card.yaml | 4 +- .../skill-card.md | 4 +- .../SKILL.md | 2 +- .../skill-card.md | 2 +- .../SKILL.md | 2 +- .../skill-card.md | 4 +- .../SKILL.md | 4 +- .../skill-card.md | 6 +- .../SKILL.md | 6 +- .../SKILL.md | 2 +- .../skill-card.md | 2 +- .../SKILL.md | 6 +- .../card.yaml | 8 +- .../skill-card.md | 4 +- .../SKILL.md | 2 +- .../card.yaml | 10 +- .../skill-card.md | 4 +- .../SKILL.md | 2 +- .../card.yaml | 4 +- .../skill-card.md | 2 +- skills/nemo-mbridge-resiliency/SKILL.md | 2 +- skills/nemo-mbridge-resiliency/card.yaml | 4 +- skills/nemo-mbridge-resiliency/skill-card.md | 4 +- 204 files changed, 2589 insertions(+), 1959 deletions(-) delete mode 100644 docs/README.md rename docs/{bridge-tech-details.md => about/architecture.mdx} (93%) create mode 100644 docs/about/index.mdx rename docs/{releases/changelog.md => about/release-notes/changelog.mdx} (74%) create mode 100644 docs/about/release-notes/index.mdx rename docs/{releases/software-versions.md => about/release-notes/software-versions.mdx} (95%) delete mode 100644 docs/autodoc2_docstrings_parser.py delete mode 100644 docs/broken_links_false_positives.json delete mode 100644 docs/conf.py delete mode 100644 docs/documentation.md create mode 100644 docs/fern/docs.yml create mode 100644 docs/fern/fern.config.json create mode 100644 docs/fern/versions/nightly.yml create mode 100644 docs/get-started/index.mdx create mode 100644 docs/get-started/installation.mdx create mode 100644 docs/get-started/migration/index.mdx rename docs/{megatron-lm-to-megatron-bridge.md => get-started/migration/megatron-lm-to-megatron-bridge.mdx} (98%) rename docs/{nemo2-migration-guide.md => get-started/migration/nemo2-to-megatron-bridge.mdx} (87%) create mode 100644 docs/get-started/prerequisites.mdx rename docs/{bridge-guide.md => get-started/quickstart/hugging-face-conversion.mdx} (95%) create mode 100644 docs/get-started/quickstart/index.mdx delete mode 100644 docs/index.md create mode 100644 docs/index.mdx rename docs/{modelopt/distillation.md => model-optimization/distillation.mdx} (99%) rename docs/{modelopt/README.md => model-optimization/index.mdx} (63%) rename docs/{modelopt/pruning.md => model-optimization/pruning.mdx} (96%) rename docs/{modelopt/quantization.md => model-optimization/quantization.mdx} (98%) delete mode 100644 docs/models/README.md rename docs/models/bailing/{index.md => index.mdx} (54%) rename docs/models/bailing/{ling-2.md => ling-2.mdx} (97%) rename docs/models/deepseek/{deepseek-v2.md => deepseek-v2.mdx} (94%) rename docs/models/deepseek/{deepseek-v3.md => deepseek-v3.mdx} (94%) rename docs/models/deepseek/{deepseek-v4.md => deepseek-v4.mdx} (95%) delete mode 100644 docs/models/deepseek/index.md create mode 100644 docs/models/deepseek/index.mdx rename docs/models/falcon/{falcon-h1.md => falcon-h1.mdx} (97%) rename docs/models/falcon/{index.md => index.mdx} (53%) rename docs/models/gemma/{gemma.md => gemma.mdx} (97%) rename docs/models/gemma/{gemma2.md => gemma2.mdx} (94%) rename docs/models/gemma/{gemma3-vl.md => gemma3-vl.mdx} (91%) rename docs/models/gemma/{gemma3.md => gemma3.mdx} (91%) rename docs/models/gemma/{gemma4-vl.md => gemma4-vl.mdx} (83%) delete mode 100644 docs/models/gemma/index.md create mode 100644 docs/models/gemma/index.mdx rename docs/models/glm/{glm-45v.md => glm-45v.mdx} (89%) rename docs/models/glm/{glm45.md => glm45.mdx} (95%) rename docs/models/glm/{glm47.md => glm47.mdx} (97%) rename docs/models/glm/{glm5.md => glm5.mdx} (96%) delete mode 100644 docs/models/glm/index.md create mode 100644 docs/models/glm/index.mdx rename docs/models/{gpt_oss/gpt-oss.md => gpt-oss/gpt-oss.mdx} (86%) rename docs/models/{gpt_oss/index.md => gpt-oss/index.mdx} (54%) create mode 100644 docs/models/index.mdx delete mode 100644 docs/models/kimi/index.md create mode 100644 docs/models/kimi/index.mdx rename docs/models/kimi/{kimi-k2.md => kimi-k2.mdx} (96%) rename docs/models/kimi/{kimi-k25-vl.md => kimi-k25-vl.mdx} (97%) rename docs/models/llama/{index.md => index.mdx} (66%) rename docs/models/llama/{llama2.md => llama2.mdx} (97%) rename docs/models/llama/{llama3.md => llama3.mdx} (96%) rename docs/models/mimo/{index.md => index.mdx} (56%) rename docs/models/mimo/{mimo.md => mimo.mdx} (96%) delete mode 100644 docs/models/minimax/index.md create mode 100644 docs/models/minimax/index.mdx rename docs/models/minimax/{minimax-m2.md => minimax-m2.mdx} (97%) delete mode 100644 docs/models/mistral/index.md create mode 100644 docs/models/mistral/index.mdx rename docs/models/mistral/{ministral3.md => ministral3.mdx} (91%) rename docs/models/mistral/{mistral.md => mistral.mdx} (91%) rename docs/models/moonlight/{index.md => index.mdx} (75%) rename docs/models/moonlight/{moonlight.md => moonlight.mdx} (93%) delete mode 100644 docs/models/nemotron/index.md create mode 100644 docs/models/nemotron/index.mdx rename docs/models/nemotron/{llama-nemotron.md => llama-nemotron.mdx} (94%) rename docs/models/nemotron/{nemotron-3-omni.md => nemotron-3-omni.mdx} (87%) rename docs/models/nemotron/{nemotron-nano-v2-vl.md => nemotron-nano-v2-vl.mdx} (98%) rename docs/models/nemotron/{nemotron3-nano.md => nemotron3-nano.mdx} (95%) rename docs/models/nemotron/{nemotron3-super.md => nemotron3-super.mdx} (97%) rename docs/models/nemotron/{nemotronh.md => nemotronh.mdx} (96%) rename docs/models/olmoe/{index.md => index.mdx} (77%) rename docs/models/olmoe/{olmoe.md => olmoe.mdx} (94%) delete mode 100644 docs/models/qwen/index.md create mode 100644 docs/models/qwen/index.mdx rename docs/models/qwen/{qwen.md => qwen.mdx} (97%) rename docs/models/qwen/{qwen2-audio.md => qwen2-audio.mdx} (96%) rename docs/models/qwen/{qwen2.5-vl.md => qwen2.5-vl.mdx} (98%) rename docs/models/qwen/{qwen25-omni.md => qwen25-omni.mdx} (96%) rename docs/models/qwen/{qwen3-asr.md => qwen3-asr.mdx} (96%) rename docs/models/qwen/{qwen3-moe.md => qwen3-moe.mdx} (93%) rename docs/models/qwen/{qwen3-next.md => qwen3-next.mdx} (94%) rename docs/models/qwen/{qwen3-omni.md => qwen3-omni.mdx} (85%) rename docs/models/qwen/{qwen3-vl.md => qwen3-vl.mdx} (94%) rename docs/models/qwen/{qwen35-vl.md => qwen35-vl.mdx} (89%) rename docs/models/sarvam/{index.md => index.mdx} (55%) rename docs/models/sarvam/{sarvam.md => sarvam.mdx} (96%) rename docs/models/stepfun/{index.md => index.mdx} (53%) rename docs/models/stepfun/{step35.md => step35.mdx} (97%) delete mode 100644 docs/project.json create mode 100644 docs/reference/api/index.mdx create mode 100644 docs/reference/index.mdx rename docs/{performance-summary-archive.md => reference/performance/archive.mdx} (99%) create mode 100644 docs/reference/performance/index.mdx rename docs/{performance-summary.md => reference/performance/summary.mdx} (99%) delete mode 100644 docs/releases/README.md create mode 100644 docs/resources/agent-skills.mdx rename docs/{releases/cherrypick-policy.md => resources/contributing/cherrypick-policy.mdx} (93%) create mode 100644 docs/resources/contributing/index.mdx rename docs/{releases/release-process.md => resources/contributing/release-process.mdx} (96%) create mode 100644 docs/resources/documentation-development.mdx create mode 100644 docs/resources/documentation-map.mdx create mode 100644 docs/resources/index.mdx delete mode 100644 docs/skills-index.md delete mode 100644 docs/training/README.md rename docs/training/{activation-recomputation.md => activation-recomputation.mdx} (91%) rename docs/training/{attention-optimizations.md => attention-optimizations.mdx} (99%) rename docs/training/{callbacks.md => callbacks.mdx} (92%) rename docs/training/{checkpointing.md => checkpointing.mdx} (96%) rename docs/training/{communication-overlap.md => communication-overlap.mdx} (83%) rename docs/training/{config-container-overview.md => config-container-overview.mdx} (99%) rename docs/training/{cpu-offloading.md => cpu-offloading.mdx} (88%) rename docs/training/{cuda-graphs.md => cuda-graphs.mdx} (93%) create mode 100644 docs/training/data-preparation.mdx rename docs/training/{entry-points.md => entry-points.mdx} (90%) rename docs/training/{hierarchical-context-parallel.md => hierarchical-context-parallel.mdx} (85%) create mode 100644 docs/training/index.mdx rename docs/training/{logging.md => logging.mdx} (93%) rename docs/training/{megatron-fsdp.md => megatron-fsdp.mdx} (86%) rename docs/training/{memory-estimator.md => memory-estimator.mdx} (92%) rename docs/training/{mixed-precision.md => mixed-precision.mdx} (90%) rename docs/training/{moe-optimization.md => moe-optimization.mdx} (93%) rename docs/training/{multi-token-prediction.md => multi-token-prediction.mdx} (92%) rename docs/training/{optimizer-scheduler.md => optimizer-scheduler.mdx} (98%) rename docs/training/{packed-sequences.md => packed-sequences.mdx} (80%) rename docs/{parallelisms.md => training/parallelisms.mdx} (96%) rename docs/training/{peft.md => peft.mdx} (97%) rename docs/{performance-guide.md => training/performance-tuning.mdx} (99%) rename docs/training/{profiling.md => profiling.mdx} (90%) rename docs/{recipe-usage.md => training/recipe-usage.mdx} (96%) rename docs/training/{resiliency.md => resiliency.mdx} (90%) rename docs/training/{training-loop-settings.md => training-loop-settings.mdx} (93%) create mode 100644 docs/troubleshooting/index.mdx rename docs/{releases/known-issues.md => troubleshooting/known-issues.mdx} (97%) create mode 100644 docs/use-cases/index.mdx rename docs/{adding-new-models.md => use-cases/model-development/add-new-model.mdx} (88%) create mode 100644 docs/use-cases/model-development/index.mdx create mode 100644 docs/use-cases/reinforcement-learning/index.mdx rename docs/{bridge-rl-integration.md => use-cases/reinforcement-learning/rl-framework-integration.mdx} (97%) delete mode 100644 docs/versions1.json diff --git a/.github/ISSUE_TEMPLATE/model_support_request.yml b/.github/ISSUE_TEMPLATE/model_support_request.yml index c83d289223..f79f3967ba 100644 --- a/.github/ISSUE_TEMPLATE/model_support_request.yml +++ b/.github/ISSUE_TEMPLATE/model_support_request.yml @@ -11,7 +11,7 @@ body: value: | Use this form to request support for a new model. Include the upstream model reference and the minimum deliverables needed to make the request actionable. - **We encourage you to try adding model support yourself first!** Megatron Bridge includes an [AI-assisted model onboarding guide](../../docs/adding-new-models.md) that walks you through creating bridges, providers, recipes, and tests. Many models share architectural patterns with already-supported families, making self-service onboarding straightforward. If you have a working implementation, please open a PR โ€” we'll help review and merge it. + **We encourage you to try adding model support yourself first!** Megatron Bridge includes an [AI-assisted model onboarding guide](../../docs/use-cases/model-development/add-new-model.mdx) that walks you through creating bridges, providers, recipes, and tests. Many models share architectural patterns with already-supported families, making self-service onboarding straightforward. If you have a working implementation, please open a PR โ€” we'll help review and merge it. - type: input id: hf-repo attributes: diff --git a/README.md b/README.md index 6b7a09d716..c5c8d373e3 100644 --- a/README.md +++ b/README.md @@ -210,23 +210,23 @@ Megatron Bridge provides out-of-the-box bridges and training recipes for a wide | Family | Supported variants | |----------------|--------------------| -| [**Bailing**](docs/models/bailing/index.md) | Ling 2.0 (Bailing) | -| [**DeepSeek**](docs/models/deepseek/index.md) | DeepSeek V2 / V2 Lite, DeepSeek V3, DeepSeek V4 | -| [**Falcon**](docs/models/falcon/index.md) | Falcon H1 | -| [**Gemma**](docs/models/gemma/index.md) | Gemma / Gemma 2, Gemma 3, Gemma 3-VL, Gemma 4-VL (26B-A4B MoE) | -| [**GLM**](docs/models/glm/index.md) | GLM-4.5 / 4.7 / 4.7-Flash, GLM-4.5V, GLM-5 / 5.1 | -| [**GPT-OSS**](docs/models/gpt_oss/index.md) | GPT-oss | -| [**Kimi**](docs/models/kimi/index.md) | Kimi K2, Kimi-K2.5-VL | -| [**Llama**](docs/models/llama/index.md) | Llama 2, Llama 3 / 3.1 / 3.2 / 3.3 | -| [**MiniMax**](docs/models/minimax/index.md) | MiniMax-M2 / M2.5 / M2.7 | -| [**Mistral**](docs/models/mistral/index.md) | Mistral, Ministral 3 (3B/8B/14B) | -| [**Xiaomi-MiMo**](docs/models/mimo/index.md) | Xiaomi-MiMo | -| [**Moonlight**](docs/models/moonlight/index.md) | Moonlight | -| [**Nemotron**](docs/models/nemotron/index.md) | Nemotron H, Nemotron Nano v2, Nemotron-3 Nano, Nemotron-3 Super, Llama Nemotron, Nemotron Nano v2 VL, Nemotron-3 Nano Omni | -| [**OLMoE**](docs/models/olmoe/index.md) | OLMoE | -| [**Qwen**](docs/models/qwen/index.md) | Qwen2 / Qwen2.5, Qwen3, Qwen3-MoE, Qwen3 Next, Qwen2.5-VL, Qwen3-VL, Qwen3.5-VL, Qwen3.6-VL, Qwen2 Audio, Qwen2.5-Omni, Qwen3-Omni, Qwen3-ASR | -| [**Sarvam**](docs/models/sarvam/index.md) | Sarvam | -| [**StepFun**](docs/models/stepfun/index.md) | Step-3.5-Flash | +| [**Bailing**](docs/models/bailing/index.mdx) | Ling 2.0 (Bailing) | +| [**DeepSeek**](docs/models/deepseek/index.mdx) | DeepSeek V2 / V2 Lite, DeepSeek V3, DeepSeek V4 | +| [**Falcon**](docs/models/falcon/index.mdx) | Falcon H1 | +| [**Gemma**](docs/models/gemma/index.mdx) | Gemma / Gemma 2, Gemma 3, Gemma 3-VL, Gemma 4-VL (26B-A4B MoE) | +| [**GLM**](docs/models/glm/index.mdx) | GLM-4.5 / 4.7 / 4.7-Flash, GLM-4.5V, GLM-5 / 5.1 | +| [**GPT-OSS**](docs/models/gpt-oss/index.mdx) | GPT-oss | +| [**Kimi**](docs/models/kimi/index.mdx) | Kimi K2, Kimi-K2.5-VL | +| [**Llama**](docs/models/llama/index.mdx) | Llama 2, Llama 3 / 3.1 / 3.2 / 3.3 | +| [**MiniMax**](docs/models/minimax/index.mdx) | MiniMax-M2 / M2.5 / M2.7 | +| [**Mistral**](docs/models/mistral/index.mdx) | Mistral, Ministral 3 (3B/8B/14B) | +| [**Xiaomi-MiMo**](docs/models/mimo/index.mdx) | Xiaomi-MiMo | +| [**Moonlight**](docs/models/moonlight/index.mdx) | Moonlight | +| [**Nemotron**](docs/models/nemotron/index.mdx) | Nemotron H, Nemotron Nano v2, Nemotron-3 Nano, Nemotron-3 Super, Llama Nemotron, Nemotron Nano v2 VL, Nemotron-3 Nano Omni | +| [**OLMoE**](docs/models/olmoe/index.mdx) | OLMoE | +| [**Qwen**](docs/models/qwen/index.mdx) | Qwen2 / Qwen2.5, Qwen3, Qwen3-MoE, Qwen3 Next, Qwen2.5-VL, Qwen3-VL, Qwen3.5-VL, Qwen3.6-VL, Qwen2 Audio, Qwen2.5-Omni, Qwen3-Omni, Qwen3-ASR | +| [**Sarvam**](docs/models/sarvam/index.mdx) | Sarvam | +| [**StepFun**](docs/models/stepfun/index.mdx) | Step-3.5-Flash | ### Launching Recipes diff --git a/docs/README.md b/docs/README.md deleted file mode 100644 index 8bfebb6ab8..0000000000 --- a/docs/README.md +++ /dev/null @@ -1,269 +0,0 @@ -# Megatron Bridge Documentation - -Welcome to the Megatron Bridge documentation! This guide helps you navigate our comprehensive documentation to find exactly what you need for training, converting, and working with language, vision-language, audio, and multimodal models. - -## ๐Ÿš€ Quick Start Paths - -### I want to - -**๐Ÿƒโ€โ™‚๏ธ Get started with model conversion** -โ†’ Start with [Bridge Guide](bridge-guide.md) for Hugging Face โ†” Megatron conversion - -**โšก Understand parallelisms and performance** -โ†’ Jump to [Parallelisms Guide](parallelisms.md) and [Performance Guide](performance-guide.md) - -**๐Ÿš€ Start training a model** -โ†’ See [Training Documentation](training/README.md) for comprehensive training guides - -**๐Ÿ“š Find model documentation** -โ†’ Browse [Supported Models](models/README.md), organized by family and model variant - -**๐Ÿ”ง Migrate from NeMo 2 or Megatron-LM** -โ†’ Check [NeMo 2 Migration Guide](nemo2-migration-guide.md) or [Megatron-LM Migration Guide](megatron-lm-to-megatron-bridge.md) - -**๐Ÿ“Š Use training recipes** -โ†’ Read [Recipe Usage](recipe-usage.md) for pre-configured training recipes and browse the repository -[`tutorials/`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/tutorials) directory for runnable data prep, -recipe, and training examples - -**๐Ÿ”Œ Add support for a new model** -โ†’ Refer to [Adding New Models](adding-new-models.md) - -**๐Ÿ“‹ Check version information** -โ†’ See [Releases Documentation](releases/README.md) for versions, changelog, and known issues - ---- - -## ๐Ÿ‘ฅ Documentation by Role - -### For ML Engineers & Researchers - -- **Start here:** [Bridge Guide](bridge-guide.md) โ†’ [Training Documentation](training/README.md) -- **Deep dive:** [Performance Guide](performance-guide.md) โ†’ [Training Optimization Guides](training/README.md#optimization-and-performance) -- **Model support:** [Supported Models](models/README.md) โ†’ [Adding New Models](adding-new-models.md) - -### For Training Engineers - -- **Start here:** [Training Documentation](training/README.md) โ†’ [Configuration Container Overview](training/config-container-overview.md) -- **Performance:** [Performance Guide](performance-guide.md) โ†’ [Performance Summary](performance-summary.md) -- **Parallelisms:** [Parallelisms Guide](parallelisms.md) โ†’ [Training Optimization](training/README.md#optimization-and-performance) - -### For Model Developers - -- **Start here:** [Bridge Guide](bridge-guide.md) โ†’ [Bridge Tech Details](bridge-tech-details.md) -- **Model support:** [Adding New Models](adding-new-models.md) โ†’ [Model Documentation](models/README.md) -- **Integration:** [Bridge RL Integration](bridge-rl-integration.md) - -### For DevOps & Platform Teams - -- **Start here:** [Releases Documentation](releases/README.md) โ†’ [Software Versions](releases/software-versions.md) -- **Troubleshooting:** [Known Issues](releases/known-issues.md) -- **API Reference:** [API Documentation](apidocs/index.rst) - ---- - -## ๐Ÿ“š Complete Documentation Index - -### Getting Started - -| Document | Purpose | When to Read | -|----------|---------|--------------| -| **[Bridge Guide](bridge-guide.md)** | Hugging Face โ†” Megatron conversion guide | First time converting models | -| **[Bridge Tech Details](bridge-tech-details.md)** | Technical details of the bridge system | Understanding bridge internals | -| **[Parallelisms Guide](parallelisms.md)** | Data and model parallelism strategies | Setting up distributed training | -| **[Performance Summary](performance-summary.md)** | Quick performance reference | Quick performance lookup | -| **[Performance Guide](performance-guide.md)** | Comprehensive performance optimization | Optimizing training performance | - -### Model Support - -| Document | Purpose | When to Read | -|----------|---------|--------------| -| **[Supported Models](models/README.md)** | Family-organized model documentation | Working with supported model variants | -| **[Adding New Models](adding-new-models.md)** | Guide for adding model support | Extending model support | - -### Training and Customization - -| Document | Purpose | When to Read | -|----------|---------|--------------| -| **[Training Documentation](training/README.md)** | Comprehensive training guides | Setting up and customizing training | -| **[Configuration Container Overview](training/config-container-overview.md)** | Central training configuration | Understanding training configuration | -| **[Entry Points](training/entry-points.md)** | Training entry points and execution | Understanding training flow | -| **[Training Loop Settings](training/training-loop-settings.md)** | Training loop parameters | Configuring training parameters | -| **[Optimizer & Scheduler](training/optimizer-scheduler.md)** | Optimization configuration | Setting up optimizers | -| **[Mixed Precision](training/mixed-precision.md)** | Mixed precision training | Reducing memory usage | -| **[PEFT](training/peft.md)** | Parameter-efficient fine-tuning | Fine-tuning with limited resources | -| **[Checkpointing](training/checkpointing.md)** | Checkpoint management | Saving and resuming training | -| **[Logging](training/logging.md)** | Logging and monitoring | Monitoring training progress | -| **[Profiling](training/profiling.md)** | Performance profiling | Identifying bottlenecks | - -### Recipes and Workflows - -| Document | Purpose | When to Read | -|----------|---------|--------------| -| **[Recipe Usage](recipe-usage.md)** | Using pre-configured training recipes | Quick training setup | -| **[Bridge RL Integration](bridge-rl-integration.md)** | Reinforcement learning integration | RL training workflows | - -### Migration Guides - -| Document | Purpose | When to Read | -|----------|---------|--------------| -| **[NeMo 2 Migration Guide](nemo2-migration-guide.md)** | Migrating from NeMo 2 | Upgrading from NeMo 2 | -| **[Megatron-LM Migration Guide](megatron-lm-to-megatron-bridge.md)** | Migrating from Megatron-LM | Upgrading from Megatron-LM | - -### Reference - -| Document | Purpose | When to Read | -|----------|---------|--------------| -| **[API Documentation](apidocs/index.rst)** | Complete API reference | Building integrations | -| **[Releases Documentation](releases/README.md)** | Version history and known issues | Checking versions, troubleshooting | -| **[Documentation Guide](documentation.md)** | Contributing to documentation | Contributing docs | - ---- - -## ๐Ÿ—บ๏ธ Common Reading Paths - -### ๐Ÿ†• First-Time Users - -1. [Bridge Guide](bridge-guide.md) *(10 min - understand conversion)* -2. [Parallelisms Guide](parallelisms.md) *(15 min - understand distributed training)* -3. [Training Documentation](training/README.md) *(choose your training path)* -4. [Recipe Usage](recipe-usage.md) *(5 min - use pre-configured recipes)* - -### ๐Ÿ”ง Setting Up Training - -1. [Training Documentation](training/README.md) *(overview of training system)* -2. [Configuration Container Overview](training/config-container-overview.md) *(understand configuration)* -3. [Entry Points](training/entry-points.md) *(how training starts)* -4. [Training Loop Settings](training/training-loop-settings.md) *(configure parameters)* -5. [Logging](training/logging.md) *(set up monitoring)* - -### โšก Performance Optimization - -1. [Performance Guide](performance-guide.md) *(comprehensive optimization strategies)* -2. [Performance Summary](performance-summary.md) *(quick reference)* -3. [Mixed Precision](training/mixed-precision.md) *(reduce memory usage)* -4. [Communication Overlap](training/communication-overlap.md) *(optimize distributed training)* -5. [Activation Recomputation](training/activation-recomputation.md) *(reduce memory footprint)* -6. [Profiling](training/profiling.md) *(identify bottlenecks)* - -### ๐Ÿ”„ Model Conversion Workflow - -1. [Bridge Guide](bridge-guide.md) *(conversion basics)* -2. [Bridge Tech Details](bridge-tech-details.md) *(technical details)* -3. [Supported Models](models/README.md) *(model-specific guides)* -4. [Adding New Models](adding-new-models.md) *(extend support)* - -### ๐Ÿ”ง Customization and Extension - -1. [Training Documentation](training/README.md) *(training customization)* -2. [PEFT](training/peft.md) *(parameter-efficient fine-tuning)* -3. [Adding New Models](adding-new-models.md) *(add model support)* -4. [Bridge RL Integration](bridge-rl-integration.md) *(RL workflows)* - -### ๐Ÿช„ Model Optimization (ModelOpt) - -1. [Model Optimization Overview](modelopt/README.md) *(quantization, distillation, pruning)* -2. [Quantization](modelopt/quantization.md) *(PTQ and QAT)* -3. [Distillation](modelopt/distillation.md) *(knowledge distillation)* -4. [Pruning](modelopt/pruning.md) *(Minitron-based pruning)* - -### ๐Ÿ“ฆ Migration Paths - -1. [NeMo 2 Migration Guide](nemo2-migration-guide.md) *(from NeMo 2)* -2. [Megatron-LM Migration Guide](megatron-lm-to-megatron-bridge.md) *(from Megatron-LM)* -3. [Training Documentation](training/README.md) *(new training system)* - ---- - -## ๐Ÿ“ Directory Structure - -### Main Documentation - -- **Guides** - Core guides for parallelisms, performance, recipes, and migration -- **Bridge Documentation** - Hugging Face โ†” Megatron conversion guides -- **Model Documentation** - Supported model families and architectures - -### Subdirectories - -#### [models/](models/README.md) - -- Family folders such as **[qwen/](models/qwen/index.md)**, **[gemma/](models/gemma/index.md)**, and **[nemotron/](models/nemotron/index.md)** - - Model-variant guides for conversion, training recipes, and architecture notes - - Multimodal variants live with their model families - -#### [training/](training/README.md) - -- **Configuration** - ConfigContainer, entry points, training loop settings -- **Optimization** - Optimizer, scheduler, mixed precision, communication overlap -- **Performance** - Attention optimizations, activation recomputation, CPU offloading -- **Monitoring** - Logging, profiling, checkpointing, resiliency -- **Advanced** - PEFT, packed sequences - -#### [modelopt/](modelopt/README.md) - -- **Quantization** - PTQ and QAT workflows (FP8, INT8, INT4) -- **Distillation** - Knowledge distillation from teacher to student -- **Pruning** - Minitron-based depth and width pruning - -#### [releases/](releases/README.md) - -- **Software Versions** - Current versions and dependencies -- **Changelog** - Release history and changes -- **Known Issues** - Bugs, limitations, and workarounds - ---- - -## ๐Ÿ”— How Documents Connect - -```mermaid -graph TD - A[README.md
Start Here] --> B[Bridge Guide
Model Conversion] - A --> C[Training Docs
Training Setup] - A --> D[Models
Model Support] - - B --> E[Bridge Tech Details
Technical Deep Dive] - B --> F[Supported Models
Model-Specific Guides] - - C --> G[Config Container
Configuration] - C --> H[Performance Guide
Optimization] - C --> I[Parallelisms
Distributed Training] - - G --> J[Training Loop
Training Parameters] - G --> K[Optimizer & Scheduler
Optimization Setup] - - H --> L[Mixed Precision
Memory Efficiency] - H --> M[Communication Overlap
Performance] - - I --> N[Data Parallelism
DDP] - I --> O[Model Parallelism
TP/PP/VPP] - - D --> P[Family Index
Model Variants] - D --> Q[Model Pages
Recipes and Conversion] - - style A fill:#e1f5fe - style B fill:#f3e5f5 - style C fill:#e8f5e8 - style D fill:#fff3e0 - style H fill:#fce4ec - style I fill:#e0f2f1 -``` - ---- - -## ๐Ÿค Getting Help - -- **GitHub Issues:** [Report bugs or request features](https://github.com/NVIDIA-NeMo/Megatron-Bridge/issues) -- **Documentation Issues:** Found something unclear? Let us know! -- **Community:** Join discussions and share experiences - ---- - -## ๐Ÿ“– Additional Resources - -- **[Examples](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples)** - Code examples and tutorials -- **[Contributing Guide](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/CONTRIBUTING.md)** - How to contribute to the project -- **[API Documentation](apidocs/index.rst)** - Complete API reference - ---- - -**Ready to get started?** Choose your path above or dive into the [Bridge Guide](bridge-guide.md) for model conversion! ๐Ÿš€ diff --git a/docs/bridge-tech-details.md b/docs/about/architecture.mdx similarity index 93% rename from docs/bridge-tech-details.md rename to docs/about/architecture.mdx index 19403c47d3..c329212c52 100644 --- a/docs/bridge-tech-details.md +++ b/docs/about/architecture.mdx @@ -1,10 +1,15 @@ +--- +title: "Megatron Bridge Conversion Technical Details" +description: "" +--- + # Megatron Bridge Conversion Technical Details Megatron Bridge provides a robust, parallelism-aware pathway to convert models and checkpoints between ๐Ÿค— Hugging Face Transformers and Megatron-Core formats. This page dives into the architecture, data flows, and per-parameter conversion engine, with examples. Megatron Bridge performs on-the-fly, model-parallel-aware, per-parameter conversionโ€”unlike traditional converters that require a single GPU and full in-memory loading of both Megatron-Core and HF models. -- For API-centric usage, see the guide: [Bridge with ๐Ÿค— Hugging Face](./bridge-guide.md) +- For API-centric usage, see the guide: [Bridge with ๐Ÿค— Hugging Face](/get-started/quickstart/hugging-face-conversion) ## Architecture at a glance @@ -24,7 +29,7 @@ flowchart LR Key components: -- AutoBridge: Detects HF architecture, constructs the appropriate bridge, exposes high-level convert/save APIs. See {doc}`apidocs/bridge/bridge.models.conversion.auto_bridge`. +- AutoBridge: Detects HF architecture, constructs the appropriate bridge, exposes high-level convert/save APIs. See [bridge.models.conversion.auto bridge](/reference/api). - MegatronModelBridge: Orchestrates conversion, builds conversion tasks, handles per-parameter streaming. See [model_bridge.py](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/src/megatron/bridge/models/conversion/model_bridge.py). - MegatronMappingRegistry: Registry of parameter-name mappings; resolves concrete `MegatronParamMapping` for each weight. See [mapping_registry.py](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/src/megatron/bridge/models/conversion/mapping_registry.py). - Param Mappings: Implement parameter transformations and parallel distribution (Auto, ColumnParallel, RowParallel, QKV, GatedMLP, Replicated, custom). See [param_mapping.py](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/src/megatron/bridge/models/conversion/param_mapping.py). @@ -67,15 +72,15 @@ registry = MegatronMappingRegistry( v="model.layers.*.self_attn.v_proj.weight", ), ) -# Example: "decoder.layers.0.self_attention.linear_qkv.weight" โ†’ QKVMapping +# Example: "decoder.layers.0.self_attention.linear_qkv.weight" -> QKVMapping ``` (4) Create conversion tasks: The bridge pairs every Megatron parameter with its resolved mapping and associated metadata (owning module, tensor handle, parallelism context). These per-parameter tasks become the unit of work for conversion. -(5) Execute conversions: For HFโ†’Megatron or Megatronโ†’HF, the bridge iterates tasks and calls the mappingโ€™s `hf_to_megatron` or `megatron_to_hf` routines. Conversion streams parameter-by-parameter to minimize memory. +(5) Execute conversions: For HF->Megatron or Megatron->HF, the bridge iterates tasks and calls the mappingโ€™s `hf_to_megatron` or `megatron_to_hf` routines. Conversion streams parameter-by-parameter to minimize memory. ```python -# HF โ†’ Megatron streaming import (internally iterates conversion tasks) +# HF -> Megatron streaming import (internally iterates conversion tasks) bridge.load_hf_weights(megatron_model) ``` @@ -87,7 +92,7 @@ Properties: - Parallelism aware: Distribution respects TP (tensor), PP (pipeline), VPP (virtual pipeline), and expert parallel settings. - Deterministic mapping: Names resolve via `MegatronMappingRegistry`, including wildcard support. -### HF โ†’ Megatron (import) +### HF -> Megatron (import) ```mermaid sequenceDiagram @@ -116,7 +121,7 @@ sequenceDiagram B-->>U: model with loaded weights ``` -### Megatron โ†’ HF (export) +### Megatron -> HF (export) ```mermaid sequenceDiagram @@ -162,13 +167,13 @@ Note: If you need a one-to-many or many-to-one mapping that is not covered by QK ### Example Mapping - ColumnParallelMapping: PP, TP, EP in practice -- HF โ†’ Megatron (import): +- HF -> Megatron (import): - HF tensors are available from storage to all ranks; TP rank 0 reads the full tensor and performs the split/scatter. - TP: Rank 0 splits along dim 0 into `tp_size` chunks and scatters shards to TP ranks so each rank receives a tensor matching its local parameter shape/dtype/device. - PP: No PP collectives are needed; the owning PP stage writes its shard directly. - EP: For expert parameters, each EP rank receives its local experts by name; no cross-EP collectives are required on import. -- Megatron โ†’ HF (export): +- Megatron -> HF (export): - Only the owning PP stage initially holds the local Megatron shard; it broadcasts to all PP ranks before TP gather. - PP: The owning PP stage first broadcasts the tensor to all PP ranks so every rank participates in the collectives. - TP: All TP shards are gathered and concatenated along dim 0 to reconstruct the full tensor. @@ -186,11 +191,9 @@ Implementation notes (from code): Embedded from `src/megatron/bridge/models/qwen/qwen3_bridge.py`: -```{literalinclude} ../src/megatron/bridge/models/qwen/qwen3_bridge.py -:language: python -:pyobject: Qwen3Bridge -:linenos: -``` + +For the latest implementation, see [qwen3_bridge.py](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/models/qwen/qwen3_bridge.py). + Notes: diff --git a/docs/about/index.mdx b/docs/about/index.mdx new file mode 100644 index 0000000000..ed0c111fab --- /dev/null +++ b/docs/about/index.mdx @@ -0,0 +1,59 @@ +--- +title: "Overview" +description: "Overview of Megatron Bridge, including core capabilities, target users, and common use cases." +--- + +# Megatron Bridge Overview + +Megatron Bridge connects Hugging Face Transformers and Megatron Core for scalable model conversion, training, fine-tuning, and checkpoint export. It provides architecture-specific bridges, model providers, training recipes, and performance tooling for language, vision-language, audio, and multimodal models. + +Megatron Bridge is designed for ML engineers, researchers, and platform teams that need reproducible large-model workflows across single-node development and multi-node training environments. + +## Core Building Blocks + +| Building Block | Description | +| --- | --- | +| Model bridges | Convert Hugging Face model configs and weights into Megatron-compatible providers and parameter mappings. | +| Training stack | Runs pretraining, SFT, PEFT, checkpointing, logging, and profiling with a PyTorch-native training loop. | +| Model recipes | Provides model-family-specific training and fine-tuning configurations. | +| Model optimization | Integrates ModelOpt workflows for quantization, distillation, and pruning. | +| Supported model catalog | Documents conversion, inference, and training notes for supported model families. | + +## Benefits + +- Use high-level conversion APIs while preserving model-parallel checkpoint handling. +- Configure scalable training with tensor, pipeline, context, expert, and data parallelism. +- Start from tested recipes for common model families and customize them for production workloads. +- Export trained Megatron checkpoints back to Hugging Face-compatible artifacts for downstream use. + +## Use Cases + +| Use Case | Start Here | +| --- | --- | +| Convert Hugging Face models to Megatron Core | [Hugging Face Conversion Quickstart](/get-started/quickstart/hugging-face-conversion) | +| Train or fine-tune supported models | [Training and Customization](/training) | +| Add a new model family | [Add a New Model](/use-cases/model-development/add-new-model) | +| Optimize a model for deployment | [Model Optimization](/model-optimization) | +| Plan an upgrade | [Release Notes](/about/release-notes) | + +## Learn More + + + + +Review the model conversion architecture and parameter mapping workflow. + + + +Choose an installation, quickstart, or migration path. + + + +Find model-family-specific conversion and training guidance. + + + +Review software versions, release process, changelog links, and known issues. + + + diff --git a/docs/releases/changelog.md b/docs/about/release-notes/changelog.mdx similarity index 74% rename from docs/releases/changelog.md rename to docs/about/release-notes/changelog.mdx index 4f72a984f8..7781076640 100644 --- a/docs/releases/changelog.md +++ b/docs/about/release-notes/changelog.mdx @@ -1,3 +1,8 @@ +--- +title: "Changelog" +description: "" +--- + # Changelog Please see the changelog details in the [Github releases](https://github.com/NVIDIA-NeMo/Megatron-Bridge/releases). diff --git a/docs/about/release-notes/index.mdx b/docs/about/release-notes/index.mdx new file mode 100644 index 0000000000..e17f6b8325 --- /dev/null +++ b/docs/about/release-notes/index.mdx @@ -0,0 +1,88 @@ +--- +title: "Releases" +description: "" +--- + +# Releases + +This directory contains release information, version history, and known issues for Megatron Bridge. + +## Release Documentation + +| Document | Purpose | When to Read | +|----------|---------|--------------| +| **[Release Process](/resources/contributing/release-process)** | Release cycle, RC cadence, code-freeze, golden values, CI annotations | Participating in or understanding a release | +| **[Cherrypick Policy](/resources/contributing/cherrypick-policy)** | Rules for what may land on release branches during code-freeze and absolute freeze | Cherry-picking or reviewing during a release window | +| **[Software Versions](/about/release-notes/software-versions)** | Current software versions and dependencies | Checking compatibility, planning upgrades | +| **[Changelog](https://github.com/NVIDIA-NeMo/Megatron-Bridge/releases)** | Detailed release history and changes | Understanding what changed in each release | +| **[Known Issues](/troubleshooting/known-issues)** | Known bugs, limitations, and workarounds | Troubleshooting issues, planning workarounds | + +## Quick Navigation + +### I want to + +**๐Ÿ“‹ Check current versions** +-> See [Software Versions](/about/release-notes/software-versions) for current versions and dependencies + +**๐Ÿ“ Review release history** +-> Read [Changelog](https://github.com/NVIDIA-NeMo/Megatron-Bridge/releases) for detailed release notes + +**๐Ÿ› Find known issues** +-> Check [Known Issues](/troubleshooting/known-issues) for bugs and workarounds + +**๐Ÿ”„ Plan an upgrade** +-> Review [Changelog](https://github.com/NVIDIA-NeMo/Megatron-Bridge/releases) and [Known Issues](/troubleshooting/known-issues) before upgrading + +**๐Ÿ” Troubleshoot a problem** +-> Check [Known Issues](/troubleshooting/known-issues) for reported issues and solutions + +**๐Ÿ’ Cherry-pick during a release** +-> See [Cherrypick Policy](/resources/contributing/cherrypick-policy) for what is allowed on release branches during code-freeze and absolute freeze + +## Release Information Overview + +### Software Versions + +The [Software Versions](/about/release-notes/software-versions) document provides: + +- Current Megatron Bridge version +- Required dependency versions +- Compatible software versions +- Version compatibility matrix + +### Changelog + +The [Changelog](https://github.com/NVIDIA-NeMo/Megatron-Bridge/releases) includes: + +- Release dates and version numbers +- New features and enhancements +- Bug fixes and improvements +- Breaking changes and migration notes +- Deprecations and removals + +### Known Issues + +The [Known Issues](/troubleshooting/known-issues) document covers: + +- Known bugs and their status +- Limitations and constraints +- Workarounds and solutions +- Planned fixes and timelines + +## Related Documentation + +- **[Main Documentation](/resources/documentation-map)** - Return to main documentation +- **[Documentation Guide](/resources/documentation-development)** - Contributing to documentation + +## Staying Up to Date + +To stay informed about releases: + +1. **Check Software Versions** - Review [Software Versions](/about/release-notes/software-versions) regularly +2. **Read Changelog** - Review [Changelog](https://github.com/NVIDIA-NeMo/Megatron-Bridge/releases) for each release +3. **Monitor Known Issues** - Check [Known Issues](/troubleshooting/known-issues) for updates +4. **Follow Release Notes** - Review release announcements for important changes + +--- + +**Need version information?** Check [Software Versions](/about/release-notes/software-versions) or return to the [main documentation](/resources/documentation-map). diff --git a/docs/releases/software-versions.md b/docs/about/release-notes/software-versions.mdx similarity index 95% rename from docs/releases/software-versions.md rename to docs/about/release-notes/software-versions.mdx index 4f4b16149a..131dfec66e 100644 --- a/docs/releases/software-versions.md +++ b/docs/about/release-notes/software-versions.mdx @@ -1,3 +1,8 @@ +--- +title: "Software Component Versions" +description: "" +--- + # Software Component Versions ## NeMo Framework 26.04 @@ -18,9 +23,9 @@ | TRT-LLM | 1.2.0 | | vLLM | 0.17.1 | -```{note} + NVIDIA NeMoโ„ข Framework Training container is built on top of NVIDIA Optimized Frameworks PyTorch 26.02 container: https://docs.nvidia.com/deeplearning/frameworks/pytorch-release-notes/index.html -``` + ## NeMo Framework 26.02 @@ -42,9 +47,9 @@ NVIDIA NeMoโ„ข Framework Training container is built on top of NVIDIA Optimized | TRT-LLM | 1.1.0 | | vLLM | 0.14.1 | -```{note} + NVIDIA NeMoโ„ข Framework Training container is built on top of NVIDIA Optimized Frameworks PyTorch 25.06 container: https://docs.nvidia.com/deeplearning/frameworks/pytorch-release-notes/index.html -``` + ## NeMo Framework 25.11 @@ -65,9 +70,9 @@ NVIDIA NeMoโ„ข Framework Training container is built on top of NVIDIA Optimized | cuDNN | 9.13.1.26 | | TRT-LLM | 1.1.0a0 | -```{note} + NVIDIA NeMoโ„ข Framework Training container is built on top of NVIDIA Optimized Frameworks PyTorch 25.06 container: https://docs.nvidia.com/deeplearning/frameworks/pytorch-release-notes/index.html -``` + ## NeMo Framework 25.09 @@ -90,6 +95,6 @@ NVIDIA NeMoโ„ข Framework Training container is built on top of NVIDIA Optimized | cuDNN | 9.11.0.98 | | TRT-LLM (supported but not included) | 1.0.0 | -```{note} + NVIDIA NeMoโ„ข Framework Training container is built on top of NVIDIA Optimized Frameworks PyTorch 25.06 container: https://docs.nvidia.com/deeplearning/frameworks/pytorch-release-notes/index.html -``` + diff --git a/docs/autodoc2_docstrings_parser.py b/docs/autodoc2_docstrings_parser.py deleted file mode 100644 index 14b722de65..0000000000 --- a/docs/autodoc2_docstrings_parser.py +++ /dev/null @@ -1,35 +0,0 @@ -# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -from docutils import nodes -from myst_parser.parsers.sphinx_ import MystParser -from sphinx.ext.napoleon.docstring import GoogleDocstring - - -class NapoleonParser(MystParser): - """Add support for Google style docstrings.""" - - def parse(self, input_string: str, document: nodes.document) -> None: - """Parse Google style docstrings.""" - - # Get the Sphinx configuration - config = document.settings.env.config - - # Process with Google style - google_parsed = str(GoogleDocstring(input_string, config)) - - return super().parse(google_parsed, document) - - -Parser = NapoleonParser diff --git a/docs/broken_links_false_positives.json b/docs/broken_links_false_positives.json deleted file mode 100644 index a2399dfb8b..0000000000 --- a/docs/broken_links_false_positives.json +++ /dev/null @@ -1,4 +0,0 @@ -{"uri": "https://www.llama.com/models/llama-3/"} -{"uri": "https://docs.nvidia.com/megatron-core/developer-guide/latest/api-guide/context_parallel.html"} -{"uri": "https://docs.nvidia.com/megatron-core/developer-guide/latest/api-guide/dist_optimizer.html"} -{"uri": "https://docs.nvidia.com/megatron-core/developer-guide/latest/api-guide/tensor_parallel.html"} diff --git a/docs/conf.py b/docs/conf.py deleted file mode 100644 index b67b384da0..0000000000 --- a/docs/conf.py +++ /dev/null @@ -1,144 +0,0 @@ -# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# Configuration file for the Sphinx documentation builder. -# -# For the full list of built-in configuration values, see the documentation: -# https://www.sphinx-doc.org/en/master/usage/configuration.html - -import os -import shutil -import sys - - -_SKILLS_SRC = os.path.join(os.path.dirname(__file__), os.pardir, "skills") -_SKILLS_DST = os.path.join(os.path.dirname(__file__), "skills") - -if os.path.isdir(_SKILLS_SRC): - if os.path.exists(_SKILLS_DST): - shutil.rmtree(_SKILLS_DST) - shutil.copytree(_SKILLS_SRC, _SKILLS_DST) - - -# -- Project information ----------------------------------------------------- -# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information - -project = "Megatron Bridge" -copyright = "2026, NVIDIA Corporation" -author = "NVIDIA Corporation" -release = "nightly" - -# -- General configuration --------------------------------------------------- -# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration - -extensions = [ - "myst_parser", # For our markdown docs - "autodoc2", # Generates API docs - "sphinx.ext.viewcode", # For adding a link to view source code in docs - "sphinx.ext.doctest", # Allows testing in docstrings - "sphinx.ext.napoleon", # For google style docstrings - "sphinx_copybutton", # For copy button in code blocks - "sphinxcontrib.mermaid", # For mermaid diagrams -] - -templates_path = ["_templates"] -exclude_patterns = [ - "_build", - "Thumbs.db", - ".DS_Store", - "skills/linting-and-formatting/SKILL.md", - "skills/parity-testing/SKILL.md", - "skills/**/BENCHMARK.md", - "skills/**/skill-card.md", -] - -suppress_warnings = [ - "misc.highlighting_failure", # skills use Cursor-specific code reference syntax -] - -# -- Options for MyST Parser (Markdown) -------------------------------------- -# MyST Parser settings -myst_enable_extensions = [ - "dollarmath", # Enables dollar math for inline math - "amsmath", # Enables LaTeX math for display mode - "colon_fence", # Enables code blocks using ::: delimiters instead of ``` - "deflist", # Supports definition lists with term: definition format - "fieldlist", # Enables field lists for metadata like :author: Name - "tasklist", # Adds support for GitHub-style task lists with [ ] and [x] - "attrs_block", # Enables setting attributes on block elements using {#id .class key=val} -] -myst_heading_anchors = 5 # Generates anchor links for headings up to level 5 - -# Configure MyST to handle mermaid code blocks -myst_fence_as_directive = ["mermaid"] - -# -- Options for Mermaid ----------------------------------------------------- -# Configure mermaid diagrams -mermaid_version = "latest" # Use the latest version of mermaid - -# -- Options for Autodoc2 --------------------------------------------------- -sys.path.insert(0, os.path.abspath("..")) - -autodoc2_packages = [ - { - "path": "../src/megatron/bridge", # Path to your package relative to conf.py - "exclude_dirs": ["converters"], # list of directory names to exclude - } -] -autodoc2_render_plugin = "myst" # Use MyST for rendering docstrings -autodoc2_output_dir = "apidocs" # Output directory for autodoc2 (relative to docs/) -# This is a workaround that uses the parser located in autodoc2_docstrings_parser.py to allow autodoc2 to -# render google style docstrings. -# Related Issue: https://github.com/sphinx-extensions2/sphinx-autodoc2/issues/33 -autodoc2_docstring_parser_regexes = [ - (r".*", "docs.autodoc2_docstrings_parser"), -] - -# -- Options for HTML output ------------------------------------------------- -# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output - -html_theme = "nvidia_sphinx_theme" -html_theme_options = { - "switcher": { - "json_url": "../versions1.json", - "version_match": release, - }, - "icon_links": [ - { - "name": "GitHub", - "url": "https://github.com/NVIDIA-NeMo/Megatron-Bridge/", - "icon": "fa-brands fa-github", - } - ], - "extra_head": { - """ - - """ - }, - "extra_footer": { - """ - - """ - }, -} -html_extra_path = ["project.json", "versions1.json"] - -# Github links are now getting rate limited from the Github Actions -linkcheck_ignore = [ - ".*github\\.com.*", - ".*githubusercontent\\.com.*", - ".*sched\\.com.*", -] -linkcheck_retries = 10 -linkcheck_rate_limit_timeout = 600 diff --git a/docs/documentation.md b/docs/documentation.md deleted file mode 100644 index 7e06a8a206..0000000000 --- a/docs/documentation.md +++ /dev/null @@ -1,145 +0,0 @@ -# Documentation Development - -## Build the Documentation - -The following sections describe how to set up and build the Megatron Bridge documentation. - -Switch to the documentation source folder and generate HTML output. - -```sh -cd docs/ -uv run --only-group docs sphinx-build . _build/html -``` - -* The resulting HTML files are generated in a `_build/html` folder that is created under the project `docs/` folder. -* The generated python API docs are placed in `apidocs` under the `docs/` folder. - -```{NOTE} -If you encounter the error "Failed to generate package metadata for megatron-core @ directory+3rdparty/Megatron-LM," -run the command to install the necessary submodules: - -`git submodule update --init --recursive` -``` - -## Live Building - -When writing documentation, it can be helpful to serve the documentation and have it update live while you edit. - -To do so, run: - -```sh -cd docs/ -uv run --only-group docs sphinx-autobuild . _build/html --port 12345 --host 0.0.0.0 -``` - -Open a web browser and go to `http://${HOST_WHERE_SPHINX_COMMAND_RUN}:12345` to view the output. - -## Write Tests in Python Docstrings - -Any code in triple backtick blocks with the `{doctest}` directive will be tested. The format follows Python's doctest module syntax, where `>>>` indicates Python input and the following line shows the expected output. Here's an example: - -```python -def add(x: int, y: int) -> int: - """ - Adds two integers together. - - Args: - x (int): The first integer to add. - y (int): The second integer to add. - - Returns: - int: The sum of x and y. - - Examples: - ```{doctest} - >>> from megatron.bridge.made_up_package import add - >>> add(1, 2) - 3 - ``` - - """ - return x + y -``` - -## Run Tests in Python Docstrings - -You can run tests in our Python docstrings with: - -```sh -cd docs/ -uv run --only-group docs sphinx-build -b doctest . _build/doctest -``` - -## Documentation Version - -The three files below control the version switcher. Before you attempt to publish a new version of the documentation, update these files in the docs/ folder to match the latest version numbers. - -The ``version`` and ``release`` variables with your GitHub releases when publishing new versions of documentation. - -```{danger} -Latest should only be ``version`` and ``release`` variables in the main branch. -``` - -### versions1.json - -This JSON file defines the versions displayed in the switcher drop down. When adding a new version to the JSON, please make sure the ``version`` and ``url`` contain same version as your release. - -Example: - -```{code-block} json -:caption: Example: versions1.json -:emphasize-lines: 9,10 - -[ - { - "preferred": true, - "version": "latest", - "url": "https://docs.nvidia.com/nemo/megatron-bridge/latest/" - }, - { - - "version": "#.#.#", - "url": "https://docs.nvidia.com/nemo/megatron-bridge/#.#.#/" - }, - { - - "version": "0.1.0", - "url": "https://docs.nvidia.com/nemo/megatron-bridge/0.1.0/" - } -] -``` - -```{tip} -Use absolute URLs for the ``url`` variable. -``` - -## project.json - -This JSON file tells the version switcher that documentation matches the selected version in the switcher. The ``version`` should contain same version as your release. - -```{code-block} json -:caption: Example: project.json -:emphasize-lines: 3 - -{ - "name": "megatron-bridge", - "version": "#.#.#" -} -``` - -## conf.py - -The conf.py ``release`` should contain same version as your release. - -```{code-block} python -:caption: Example: conf.py -:emphasize-lines: 7 - -# -- Project information ----------------------------------------------------- -# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information - -project = "Megatron Bridge" -copyright = "2025, NVIDIA Corporation" -author = "NVIDIA Corporation" -release = "#.#.#" -``` diff --git a/docs/fern/docs.yml b/docs/fern/docs.yml new file mode 100644 index 0000000000..d336d74771 --- /dev/null +++ b/docs/fern/docs.yml @@ -0,0 +1,33 @@ +instances: + - url: megatron-bridge.docs.buildwithfern.com/nemo/megatron-bridge + custom-domain: docs.nvidia.com/nemo/megatron-bridge + multi-source: true + +title: NVIDIA NeMo Megatron Bridge + +global-theme: nvidia + +logo: + href: /nemo/megatron-bridge/nightly + right-text: Megatron Bridge + +navbar-links: + - type: github + value: https://github.com/NVIDIA-NeMo/Megatron-Bridge + +versions: + - display-name: Nightly + path: versions/nightly.yml + slug: nightly + +redirects: + - source: "/nemo/megatron-bridge/index.html" + destination: "/nemo/megatron-bridge/nightly" + - source: "/nemo/megatron-bridge/index" + destination: "/nemo/megatron-bridge/nightly" + - source: "/nemo/megatron-bridge/nightly/:path*/index.html" + destination: "/nemo/megatron-bridge/nightly/:path*" + - source: "/nemo/megatron-bridge/:path*/index.html" + destination: "/nemo/megatron-bridge/:path*" + - source: "/nemo/megatron-bridge/:path*.html" + destination: "/nemo/megatron-bridge/:path*" diff --git a/docs/fern/fern.config.json b/docs/fern/fern.config.json new file mode 100644 index 0000000000..bcc9e4d655 --- /dev/null +++ b/docs/fern/fern.config.json @@ -0,0 +1,4 @@ +{ + "organization": "nvidia", + "version": "5.38.1" +} diff --git a/docs/fern/versions/nightly.yml b/docs/fern/versions/nightly.yml new file mode 100644 index 0000000000..fd8f7c527f --- /dev/null +++ b/docs/fern/versions/nightly.yml @@ -0,0 +1,50 @@ +navigation: + - section: "Documentation" + contents: + - page: "Home" + path: ../../index.mdx + - section: "About" + contents: + - folder: ../../about + title: "Overview" + title-source: frontmatter + - section: "Get Started" + contents: + - folder: ../../get-started + title: "Getting Started" + title-source: frontmatter + - section: "Supported Models" + contents: + - folder: ../../models + title: "Supported Models" + title-source: frontmatter + - section: "Training and Customization" + contents: + - folder: ../../training + title: "Training and Customization" + title-source: frontmatter + - section: "Model Optimization" + contents: + - folder: ../../model-optimization + title: "Model Optimization" + title-source: frontmatter + - section: "Use Cases" + contents: + - folder: ../../use-cases + title: "Use Cases" + title-source: frontmatter + - section: "Reference" + contents: + - folder: ../../reference + title: "Reference" + title-source: frontmatter + - section: "Troubleshooting" + contents: + - folder: ../../troubleshooting + title: "Troubleshooting" + title-source: frontmatter + - section: "Resources" + contents: + - folder: ../../resources + title: "Resources" + title-source: frontmatter diff --git a/docs/get-started/index.mdx b/docs/get-started/index.mdx new file mode 100644 index 0000000000..a2084418f7 --- /dev/null +++ b/docs/get-started/index.mdx @@ -0,0 +1,46 @@ +--- +title: "Getting Started" +description: "Start using Megatron Bridge by reviewing prerequisites, installing the project, and running quickstart workflows." +--- + +# Getting Started + +Start here if you are installing Megatron Bridge, running your first conversion workflow, or migrating from another training stack. + +## Before You Start + +Before you begin, verify the following: + +- Access to a Linux environment or NeMo Framework container. +- Python and package tooling compatible with this repository. +- GPU access for realistic conversion, inference, or training workflows. +- Hugging Face access tokens for gated model repositories when required. + +## Start Paths + + + + +Review software, hardware, and credential requirements. + + + +Install from source or use the NeMo Framework container. + + + +Run a first conversion or training workflow. + + + +Move from NeMo 2 or Megatron-LM to Megatron Bridge. + + + + +## Typical Workflow + +1. Review [Prerequisites](/get-started/prerequisites). +2. Complete [Installation](/get-started/installation). +3. Run the [Hugging Face Conversion Quickstart](/get-started/quickstart/hugging-face-conversion). +4. Continue to [Training and Customization](/training) or [Supported Models](/models). diff --git a/docs/get-started/installation.mdx b/docs/get-started/installation.mdx new file mode 100644 index 0000000000..bec4daa8ab --- /dev/null +++ b/docs/get-started/installation.mdx @@ -0,0 +1,55 @@ +--- +title: "Installation" +description: "Install Megatron Bridge from source or use the NeMo Framework container." +--- + +# Installation + +Megatron Bridge is typically used from a source checkout or inside the NeMo Framework container. The container path provides the most complete environment for GPU training and model-family examples. + +## Installation Methods + + + + + +Start a NeMo Framework container from the NVIDIA NGC catalog and mount your checkout: + +```bash +docker run --rm -it -w /workdir -v $(pwd):/workdir \ + --entrypoint bash \ + --gpus all \ + nvcr.io/nvidia/nemo:${TAG} +``` + + + + + +Install from a local checkout for development and documentation work: + +```bash +git clone https://github.com/NVIDIA-NeMo/Megatron-Bridge.git +cd Megatron-Bridge +git submodule update --init --recursive +``` + +Follow the repository contribution guide for the current development install command. + + + + + +## Verify Access + +If you plan to use gated Hugging Face models, set `HF_TOKEN` before running conversion workflows: + +```bash +export HF_TOKEN= +``` + +## Next Steps + +- Run the [Hugging Face Conversion Quickstart](/get-started/quickstart/hugging-face-conversion). +- Browse [Supported Models](/models). +- Review [Training and Customization](/training). diff --git a/docs/get-started/migration/index.mdx b/docs/get-started/migration/index.mdx new file mode 100644 index 0000000000..af0e19fa30 --- /dev/null +++ b/docs/get-started/migration/index.mdx @@ -0,0 +1,20 @@ +--- +title: "Migration" +description: "Migration guides for moving to Megatron Bridge." +--- + +# Migration + +Use these guides when moving existing training workflows to Megatron Bridge. + + + + +Map NeMo 2 configurations, recipes, callbacks, and launch flows to Megatron Bridge. + + + +Map Megatron-LM arguments and scripts to Megatron Bridge workflows. + + + diff --git a/docs/megatron-lm-to-megatron-bridge.md b/docs/get-started/migration/megatron-lm-to-megatron-bridge.mdx similarity index 98% rename from docs/megatron-lm-to-megatron-bridge.md rename to docs/get-started/migration/megatron-lm-to-megatron-bridge.mdx index 1e4515aeb3..37450a5854 100644 --- a/docs/megatron-lm-to-megatron-bridge.md +++ b/docs/get-started/migration/megatron-lm-to-megatron-bridge.mdx @@ -1,12 +1,17 @@ +--- +title: "Megatron-LM to Megatron Bridge Guide" +description: "" +--- + # Megatron-LM to Megatron Bridge Guide -Megatron Bridge is Python-first: configure models, data, and training via typed Python APIs. All configuration lives in a structured `ConfigContainer` (see [Configuration overview](training/config-container-overview.md)). Any field can be overridden from the command line using Hydra/OmegaConf syntax in the example training scripts. +Megatron Bridge is Python-first: configure models, data, and training via typed Python APIs. All configuration lives in a structured `ConfigContainer` (see [Configuration overview](/training/config-container-overview)). Any field can be overridden from the command line using Hydra/OmegaConf syntax in the example training scripts. ## Automated config translation script `scripts/translate_mlm_to_bridge.py` translates bidirectionally between Megatron-LM `pretrain_gpt.py` CLI arguments and Megatron Bridge `run_recipe.py` Hydra overrides. It is useful for running loss-correlation experiments between the two frameworks and for migrating existing MLM configs. -### MLM โ†’ Bridge (default direction) +### MLM -> Bridge (default direction) ```bash # From a YAML config file (MODEL_ARGS section) @@ -25,7 +30,7 @@ uv run python scripts/translate_mlm_to_bridge.py \ --yaml DeepSeek-V3.yaml -o bridge_overrides.txt ``` -### Bridge โ†’ MLM (reverse direction) +### Bridge -> MLM (reverse direction) ```bash # From a Bridge recipe name (defaults exported as MLM args) @@ -91,7 +96,7 @@ uv run python scripts/translate_mlm_to_bridge.py --reverse \ Flags not present in Bridge (e.g., `--use-mcore-models`, `--use-flash-attn`) are silently skipped with a comment. `--mock-data` translates to `dataset.mock=true`. Unknown flags are listed in a separate section so you can handle them manually. -> **Activation function CLI overrides**: `model.activation_func` can now be set via Hydra CLI string override (e.g. `model.activation_func=silu`, `model.activation_func=gelu`). The string is resolved to the callable in `TransformerConfig.finalize()`. This makes `--swiglu` โ†’ `model.gated_linear_unit=true model.activation_func=silu` round-trippable from the CLI. +> **Activation function CLI overrides**: `model.activation_func` can now be set via Hydra CLI string override (e.g. `model.activation_func=silu`, `model.activation_func=gelu`). The string is resolved to the callable in `TransformerConfig.finalize()`. This makes `--swiglu` -> `model.gated_linear_unit=true model.activation_func=silu` round-trippable from the CLI. ## Quick start @@ -354,7 +359,7 @@ Additional distributed/optimizer overlap settings: | `--inprocess-barrier-timeout` | `inprocess_restart.barrier_timeout` | Timeout for internal barriers. | | `--inprocess-completion-timeout` | `inprocess_restart.completion_timeout` | Timeout for completion barrier. | | `--inprocess-last-call-wait` | `inprocess_restart.last_call_wait` | Delay to collect terminal failures. | -| `--inprocess-termination-grace-time` | `inprocess_restart.termination_grace_time` | SIGTERMโ†’SIGKILL grace period. | +| `--inprocess-termination-grace-time` | `inprocess_restart.termination_grace_time` | SIGTERM->SIGKILL grace period. | | `--inprocess-granularity` | `inprocess_restart.granularity` | Restart granularity (node/rank). | | `--inprocess-active-world-size` | `inprocess_restart.active_world_size` | Active ranks count; rest are reserve. | | `--inprocess-empty-cuda-cache` | `inprocess_restart.empty_cuda_cache` | Empty CUDA cache on restart finalize. | diff --git a/docs/nemo2-migration-guide.md b/docs/get-started/migration/nemo2-to-megatron-bridge.mdx similarity index 87% rename from docs/nemo2-migration-guide.md rename to docs/get-started/migration/nemo2-to-megatron-bridge.mdx index 18177ed354..670b5e8533 100644 --- a/docs/nemo2-migration-guide.md +++ b/docs/get-started/migration/nemo2-to-megatron-bridge.mdx @@ -1,6 +1,11 @@ +--- +title: "NeMo 2.0 to Megatron Bridge Migration Guide" +description: "" +--- + # NeMo 2.0 to Megatron Bridge Migration Guide -This guide helps you migrate from NeMo 2.0 training and recipes to Megatron Bridge. Megatron Bridge retains the Pythonic, code-first API that NeMo 2.0 developed while simplifying configuration into a single {py:class}`bridge.training.config.ConfigContainer` with typed sub-configs. Model parallelism and performance features from Megatron Core remain first-class. +This guide helps you migrate from NeMo 2.0 training and recipes to Megatron Bridge. Megatron Bridge retains the Pythonic, code-first API that NeMo 2.0 developed while simplifying configuration into a single `bridge.training.config.ConfigContainer` with typed sub-configs. Model parallelism and performance features from Megatron Core remain first-class. ## What Stays the Same @@ -22,7 +27,7 @@ Megatron Bridge derives model provider settings from Hugging Face configs throug | `llm.Llama3Config8B` | `AutoBridge.from_hf_pretrained("meta-llama/Meta-Llama-3-8B").to_megatron_provider()` | | `llm.Llama31Config70B` | `AutoBridge.from_hf_pretrained("meta-llama/Llama-3.1-70B").to_megatron_provider()` | | `llm.Qwen2Config7B` | `AutoBridge.from_hf_pretrained("Qwen/Qwen2-7B").to_megatron_provider()` | -| `llm.DeepseekV2Config` | {py:class}`bridge.models.DeepseekV2ModelProvider` | +| `llm.DeepseekV2Config` | `bridge.models.DeepseekV2ModelProvider` | ### Supported Model Families @@ -34,7 +39,7 @@ Megatron Bridge supports the following model families through bridge mappings an - **Nemotron**: Nemotron3, Nemotron4, NemotronH, NemotronNano - **NVIDIA Mamba**: Mamba variants and hybrid models -For a complete list of bridge mappings, providers, and their parameters, see {py:mod}`bridge.models`. +For a complete list of bridge mappings, providers, and their parameters, see `bridge.models`. @@ -178,7 +183,7 @@ pretrain(cfg, forward_step_func=forward_step) ### Fine-Tuning Migration Example (SFT/PEFT) -For fine-tuning, use {py:class}`bridge.training.config.FinetuningDatasetConfig` for data and set `checkpoint.pretrained_checkpoint` to the base model. Optionally add a `peft` configuration for parameter-efficient training. +For fine-tuning, use `bridge.training.config.FinetuningDatasetConfig` for data and set `checkpoint.pretrained_checkpoint` to the base model. Optionally add a `peft` configuration for parameter-efficient training. #### Before: NeMo 2.0 ```python @@ -301,7 +306,7 @@ cfg.model.tensor_model_parallel_size = 4 pretrain(cfg, forward_step_func=forward_step) ``` -For details on using and customizing recipes, see {doc}`recipe-usage`. +For details on using and customizing recipes, see [recipe usage](/training/recipe-usage). ### Migrating a Custom Recipe @@ -466,20 +471,20 @@ What used to be configured across Lightning `Trainer` arguments, callbacks, and | Configuration Area | Megatron Bridge Config Class | |-------------------|-------------------| -| Training loop settings | {py:class}`bridge.training.config.TrainingConfig` | -| Checkpointing | {py:class}`bridge.training.config.CheckpointConfig` | -| Logging and monitoring | {py:class}`bridge.training.config.LoggerConfig` | -| Distributed training initialization | {py:class}`bridge.training.config.DistributedInitConfig` | -| Mixed precision | {py:class}`bridge.training.mixed_precision.MixedPrecisionConfig` | -| Performance profiling | {py:class}`bridge.training.config.ProfilingConfig` | +| Training loop settings | `bridge.training.config.TrainingConfig` | +| Checkpointing | `bridge.training.config.CheckpointConfig` | +| Logging and monitoring | `bridge.training.config.LoggerConfig` | +| Distributed training initialization | `bridge.training.config.DistributedInitConfig` | +| Mixed precision | `bridge.training.mixed_precision.MixedPrecisionConfig` | +| Performance profiling | `bridge.training.config.ProfilingConfig` | For detailed documentation on each configuration area, see the training documentation: -- {doc}`training/config-container-overview` - Overview of the configuration system -- {doc}`training/training-loop-settings` - Training loop parameters and validation -- {doc}`training/checkpointing` - Checkpointing and model persistence -- {doc}`training/optimizer-scheduler` - Optimization and learning rate scheduling -- {doc}`training/logging` - Logging, TensorBoard, and Weights & Biases -- {doc}`training/profiling` - Performance profiling with Nsys and PyTorch +- [config container overview](/training/config-container-overview) - Overview of the configuration system +- [training loop settings](/training/training-loop-settings) - Training loop parameters and validation +- [checkpointing](/training/checkpointing) - Checkpointing and model persistence +- [optimizer scheduler](/training/optimizer-scheduler) - Optimization and learning rate scheduling +- [logging](/training/logging) - Logging, TensorBoard, and Weights & Biases +- [profiling](/training/profiling) - Performance profiling with Nsys and PyTorch @@ -488,11 +493,11 @@ Lightning `Trainer` parameters are now managed through dedicated configuration c | **Setting Category** | **NeMo 2.0 Location** | **Megatron Bridge Location** | **Details** | |---------------------|----------------------|-------------------|-------------| -| **Training iterations** | `trainer.max_steps` | {py:attr}`bridge.training.config.TrainingConfig.train_iters` | Total training iterations | -| **Validation frequency** | `trainer.val_check_interval` | {py:attr}`bridge.training.config.TrainingConfig.eval_interval` | Steps between validation runs | -| **Validation iterations** | `trainer.limit_val_batches` | {py:attr}`bridge.training.config.TrainingConfig.eval_iters` | Number of validation steps per run | -| **Test iterations** | `trainer.limit_test_batches` | {py:attr}`bridge.training.config.TrainingConfig.eval_iters` | Number of test steps (shares eval_iters) | -| **Logging frequency** | `trainer.log_every_n_steps` | {py:attr}`bridge.training.config.LoggerConfig.log_interval` | Logging frequency | +| **Training iterations** | `trainer.max_steps` | `bridge.training.config.TrainingConfig.train_iters` | Total training iterations | +| **Validation frequency** | `trainer.val_check_interval` | `bridge.training.config.TrainingConfig.eval_interval` | Steps between validation runs | +| **Validation iterations** | `trainer.limit_val_batches` | `bridge.training.config.TrainingConfig.eval_iters` | Number of validation steps per run | +| **Test iterations** | `trainer.limit_test_batches` | `bridge.training.config.TrainingConfig.eval_iters` | Number of test steps (shares eval_iters) | +| **Logging frequency** | `trainer.log_every_n_steps` | `bridge.training.config.LoggerConfig.log_interval` | Logging frequency | #### Before: NeMo 2.0 ```python @@ -518,7 +523,7 @@ logger_config = LoggerConfig(log_interval=10) # was log_every_n_steps ### Data Configuration Migration -NeMo 2.0 uses `PreTrainingDataModule` and `FineTuningDataModule` classes. Megatron Bridge uses configuration objects: {py:class}`bridge.training.config.GPTDatasetConfig` for pretraining and {py:class}`bridge.training.config.FinetuningDatasetConfig` for fine-tuning. +NeMo 2.0 uses `PreTrainingDataModule` and `FineTuningDataModule` classes. Megatron Bridge uses configuration objects: `bridge.training.config.GPTDatasetConfig` for pretraining and `bridge.training.config.FinetuningDatasetConfig` for fine-tuning. #### Pretraining Data @@ -589,8 +594,8 @@ dataset_config = GPTDatasetConfig( ``` **Key differences:** -- NeMo 2.0's `paths` โ†’ Megatron Bridge's `blend` -- NeMo 2.0's zipped list `["30", "path1", "70", "path2"]` โ†’ Megatron Bridge's separate `blend` and `blend_weights` +- NeMo 2.0's `paths` -> Megatron Bridge's `blend` +- NeMo 2.0's zipped list `["30", "path1", "70", "path2"]` -> Megatron Bridge's separate `blend` and `blend_weights` - Batch sizes move from data module to `TrainingConfig` - Dataloader options (`num_workers`, `pin_memory`, etc.) available in both configs @@ -639,7 +644,7 @@ train_config = TrainingConfig( ### Tokenizer Migration -Megatron Bridge uses {py:class}`bridge.training.tokenizers.config.TokenizerConfig` for consistent tokenizer setup across different model types. +Megatron Bridge uses `bridge.training.tokenizers.config.TokenizerConfig` for consistent tokenizer setup across different model types. #### Before: NeMo 2.0 ```python @@ -768,22 +773,22 @@ model = GPTModelProvider( ``` ### DDP Configuration Migration -Some `MegatronStrategy` parameters move to {py:class}`bridge.training.config.DistributedDataParallelConfig`: +Some `MegatronStrategy` parameters move to `bridge.training.config.DistributedDataParallelConfig`: | **Setting** | **NeMo 2.0** | **Megatron Bridge** | |-------------|-------------|-----------| -| **Distributed Optimizer Instances** | `strategy.num_distributed_optimizer_instances` | {py:attr}`bridge.training.config.DistributedDataParallelConfig.num_distributed_optimizer_instances` | +| **Distributed Optimizer Instances** | `strategy.num_distributed_optimizer_instances` | `bridge.training.config.DistributedDataParallelConfig.num_distributed_optimizer_instances` | ### Strategy Settings Migration -Additional `MegatronStrategy` parameters move to {py:class}`bridge.training.config.DistributedInitConfig`: +Additional `MegatronStrategy` parameters move to `bridge.training.config.DistributedInitConfig`: | **Setting** | **NeMo 2.0** | **Megatron Bridge** | |-------------|-------------|-----------| -| **Process Groups** | `strategy.use_gloo_process_groups` | {py:attr}`bridge.training.config.DistributedInitConfig.use_gloo_process_groups` | -| **SHARP** | `strategy.use_sharp` | {py:attr}`bridge.training.config.DistributedInitConfig.use_sharp` | -| **NCCL Config** | `strategy.nccl_communicator_config_path` | {py:attr}`bridge.training.config.DistributedInitConfig.nccl_communicator_config_path` | -| **Mapping Order** | `strategy.use_tp_pp_dp_mapping` | {py:attr}`bridge.training.config.DistributedInitConfig.use_tp_pp_dp_mapping` | -| **Lazy Init** | `strategy.lazy_init` | {py:attr}`bridge.training.config.DistributedInitConfig.lazy_init` | +| **Process Groups** | `strategy.use_gloo_process_groups` | `bridge.training.config.DistributedInitConfig.use_gloo_process_groups` | +| **SHARP** | `strategy.use_sharp` | `bridge.training.config.DistributedInitConfig.use_sharp` | +| **NCCL Config** | `strategy.nccl_communicator_config_path` | `bridge.training.config.DistributedInitConfig.nccl_communicator_config_path` | +| **Mapping Order** | `strategy.use_tp_pp_dp_mapping` | `bridge.training.config.DistributedInitConfig.use_tp_pp_dp_mapping` | +| **Lazy Init** | `strategy.lazy_init` | `bridge.training.config.DistributedInitConfig.lazy_init` | ### Mixed Precision Migration @@ -820,36 +825,36 @@ config = ConfigContainer( ) ``` -```{Note} -The `mixed_precision` configuration automatically synchronizes precision settings across model, optimizer, and DDP configurations, overriding any conflicting settings. This ensures consistent precision behavior throughout training. For details on configuration precedence and available recipes, see {doc}`training/mixed-precision`. -``` + +The `mixed_precision` configuration automatically synchronizes precision settings across model, optimizer, and DDP configurations, overriding any conflicting settings. This ensures consistent precision behavior throughout training. For details on configuration precedence and available recipes, see [mixed precision](/training/mixed-precision). + ### Checkpointing Configuration Migration -Checkpointing configuration moves from `MegatronStrategy` parameters and `ModelCheckpoint` callback to {py:class}`bridge.training.config.CheckpointConfig`: +Checkpointing configuration moves from `MegatronStrategy` parameters and `ModelCheckpoint` callback to `bridge.training.config.CheckpointConfig`: | **Checkpoint Setting** | **NeMo 2.0** | **Megatron Bridge** | |------------------------|-------------|-----------| -| **Save directory** | `ModelCheckpoint(dirpath=...)` | {py:attr}`bridge.training.config.CheckpointConfig.save` | -| **Load directory** | `trainer.ckpt_path` | {py:attr}`bridge.training.config.CheckpointConfig.load` | -| **Pretrained checkpoint (for finetuning)** | `AutoResume.import_path` or manually load | {py:attr}`bridge.training.config.CheckpointConfig.pretrained_checkpoint` | -| **Save frequency** | `ModelCheckpoint(every_n_train_steps=...)` | {py:attr}`bridge.training.config.CheckpointConfig.save_interval` | +| **Save directory** | `ModelCheckpoint(dirpath=...)` | `bridge.training.config.CheckpointConfig.save` | +| **Load directory** | `trainer.ckpt_path` | `bridge.training.config.CheckpointConfig.load` | +| **Pretrained checkpoint (for finetuning)** | `AutoResume.import_path` or manually load | `bridge.training.config.CheckpointConfig.pretrained_checkpoint` | +| **Save frequency** | `ModelCheckpoint(every_n_train_steps=...)` | `bridge.training.config.CheckpointConfig.save_interval` | | **Save top-k** | `ModelCheckpoint(save_top_k=...)` | No direct equivalent - Megatron Bridge can keep the most recent checkpoints | -| **Most recent checkpoints** | No direct equivalent | {py:attr}`bridge.training.config.CheckpointConfig.most_recent_k` | +| **Most recent checkpoints** | No direct equivalent | `bridge.training.config.CheckpointConfig.most_recent_k` | | **Save last** | `ModelCheckpoint(save_last=...)` | Always enabled in Megatron Bridge | -| **Checkpoint format** | `strategy.save_ckpt_format` | {py:attr}`bridge.training.config.CheckpointConfig.ckpt_format` | -| **Async saving** | `strategy.ckpt_async_save` | {py:attr}`bridge.training.config.CheckpointConfig.async_save` | -| **Parallel save** | `strategy.ckpt_parallel_save` | {py:attr}`bridge.training.config.CheckpointConfig.fully_parallel_save` | -| **Parallel load** | `strategy.ckpt_parallel_load` | {py:attr}`bridge.training.config.CheckpointConfig.fully_parallel_load` | -| **Load optimizer** | `strategy.ckpt_load_optimizer` | {py:attr}`bridge.training.config.CheckpointConfig.load_optim` | -| **Save optimizer** | `strategy.ckpt_save_optimizer` | {py:attr}`bridge.training.config.CheckpointConfig.save_optim` | -| **Load main params** | `strategy.ckpt_load_main_params` | {py:attr}`bridge.training.config.CheckpointConfig.load_main_params_from_ckpt` | +| **Checkpoint format** | `strategy.save_ckpt_format` | `bridge.training.config.CheckpointConfig.ckpt_format` | +| **Async saving** | `strategy.ckpt_async_save` | `bridge.training.config.CheckpointConfig.async_save` | +| **Parallel save** | `strategy.ckpt_parallel_save` | `bridge.training.config.CheckpointConfig.fully_parallel_save` | +| **Parallel load** | `strategy.ckpt_parallel_load` | `bridge.training.config.CheckpointConfig.fully_parallel_load` | +| **Load optimizer** | `strategy.ckpt_load_optimizer` | `bridge.training.config.CheckpointConfig.load_optim` | +| **Save optimizer** | `strategy.ckpt_save_optimizer` | `bridge.training.config.CheckpointConfig.save_optim` | +| **Load main params** | `strategy.ckpt_load_main_params` | `bridge.training.config.CheckpointConfig.load_main_params_from_ckpt` | | **Save weights only** | `ModelCheckpoint(save_weights_only=...)` | Inverse of `save_optim` | -| **Load strictness** | `strategy.ckpt_load_strictness` | {py:attr}`bridge.training.config.CheckpointConfig.dist_ckpt_strictness` | -| **Assume constant structure** | `strategy.ckpt_assume_constant_structure` | {py:attr}`bridge.training.config.CheckpointConfig.ckpt_assume_constant_structure` | +| **Load strictness** | `strategy.ckpt_load_strictness` | `bridge.training.config.CheckpointConfig.dist_ckpt_strictness` | +| **Assume constant structure** | `strategy.ckpt_assume_constant_structure` | `bridge.training.config.CheckpointConfig.ckpt_assume_constant_structure` | | **Save optim on train end** | `ModelCheckpoint(save_optim_on_train_end=...)` | Controlled by `save_optim` | -| **Resume from directory** | `AutoResume(resume_from_directory=...)` | {py:attr}`bridge.training.config.CheckpointConfig.load` | +| **Resume from directory** | `AutoResume(resume_from_directory=...)` | `bridge.training.config.CheckpointConfig.load` | | **Resume if exists** | `AutoResume(resume_if_exists=...)` | Automatic if `load` is set | -| **Resume ignore no checkpoint** | `AutoResume(resume_ignore_no_checkpoint=...)` | {py:attr}`bridge.training.config.CheckpointConfig.exit_on_missing_checkpoint` (inverse) | +| **Resume ignore no checkpoint** | `AutoResume(resume_ignore_no_checkpoint=...)` | `bridge.training.config.CheckpointConfig.exit_on_missing_checkpoint` (inverse) | #### Before: NeMo 2.0 ```python @@ -926,17 +931,17 @@ checkpoint_config = CheckpointConfig( - **Top-k**: NeMo 2.0's `save_top_k` monitors metrics; Megatron Bridge's `most_recent_k` keeps recent checkpoints - **Configuration location**: All checkpoint settings unified in one config (not split across callback, logger, and strategy) -```{Important} -All checkpoint paths (`save`, `load`, `pretrained_checkpoint`) must point to **Megatron-format checkpoints**. Hugging Face checkpoints cannot be used directlyโ€”convert them first using {py:meth}`bridge.models.conversion.auto_bridge.AutoBridge.import_ckpt`. See {doc}`bridge-guide` for conversion details. -``` + +All checkpoint paths (`save`, `load`, `pretrained_checkpoint`) must point to **Megatron-format checkpoints**. Hugging Face checkpoints cannot be used directlyโ€”convert them first using `bridge.models.conversion.auto_bridge.AutoBridge.import_ckpt`. See [bridge guide](/get-started/quickstart/hugging-face-conversion) for conversion details. + -For comprehensive documentation on checkpoint formats, local checkpointing, fault tolerance, and advanced features, see {doc}`training/checkpointing`. +For comprehensive documentation on checkpoint formats, local checkpointing, fault tolerance, and advanced features, see [checkpointing](/training/checkpointing). --- ### Optimizer and LR Scheduler Migration -Optimization configuration moves from NeMo 2.0's `MegatronOptimizerModule` approach to Megatron Bridge's direct {py:class}`megatron.core.optimizer.OptimizerConfig` and {py:class}`bridge.training.config.SchedulerConfig`. +Optimization configuration moves from NeMo 2.0's `MegatronOptimizerModule` approach to Megatron Bridge's direct `megatron.core.optimizer.OptimizerConfig` and `bridge.training.config.SchedulerConfig`. #### Before: NeMo 2.0 ```python @@ -997,7 +1002,7 @@ scheduler_config = SchedulerConfig( ### Logging Configuration Migration -NeMo 2.0 uses `NeMoLogger` for TensorBoard and Weights & Biases (W&B) integration. Megatron Bridge consolidates logging configuration in {py:class}`bridge.training.config.LoggerConfig`. +NeMo 2.0 uses `NeMoLogger` for TensorBoard and Weights & Biases (W&B) integration. Megatron Bridge consolidates logging configuration in `bridge.training.config.LoggerConfig`. #### Before: NeMo 2.0 @@ -1048,11 +1053,11 @@ logger_config = LoggerConfig( - Fine-grained control over what gets logged (timers, memory, validation perplexity, etc.) - No separate `NeMoLogger.setup()` call needed -For more details on logging configuration and available options, see {doc}`training/logging`. +For more details on logging configuration and available options, see [logging](/training/logging). ### Profiling Configuration Migration -Megatron Bridge centralizes all profiling functionality in {py:class}`bridge.training.config.ProfilingConfig`, replacing multiple NeMo callbacks. +Megatron Bridge centralizes all profiling functionality in `bridge.training.config.ProfilingConfig`, replacing multiple NeMo callbacks. #### Nsys Profiling Migration @@ -1176,10 +1181,10 @@ config = ConfigContainer( - Target module names are the same between NeMo 2.0 and Megatron Bridge **Supported PEFT methods:** -- **LoRA**: Low-Rank Adaptation via {py:class}`bridge.peft.lora.LoRA` -- **DoRA**: Weight-Decomposed Low-Rank Adaptation via {py:class}`bridge.peft.dora.DoRA` +- **LoRA**: Low-Rank Adaptation via `bridge.peft.lora.LoRA` +- **DoRA**: Weight-Decomposed Low-Rank Adaptation via `bridge.peft.dora.DoRA` -For comprehensive PEFT documentation including adapter design, checkpoint handling, wildcard targeting, and best practices, see {doc}`training/peft`. +For comprehensive PEFT documentation including adapter design, checkpoint handling, wildcard targeting, and best practices, see [peft](/training/peft). ## Entry Points: `pretrain` and `finetune` @@ -1273,7 +1278,7 @@ def forward_step( """ ``` -For GPT models, use the provided {py:func}`bridge.training.gpt_step.forward_step`. For custom models or specialized training logic, implement your own following this pattern. +For GPT models, use the provided `bridge.training.gpt_step.forward_step`. For custom models or specialized training logic, implement your own following this pattern. **Key differences:** - All configuration consolidated into single `ConfigContainer` object @@ -1397,11 +1402,11 @@ AutoBridge.import_ckpt( ) ``` -See {doc}`bridge-guide` for more details on model conversion. +See [bridge guide](/get-started/quickstart/hugging-face-conversion) for more details on model conversion. ### Advanced: Custom Forward Step and Loss Reduction -For comprehensive documentation on entry points, forward step functions, and customization patterns, see {doc}`training/entry-points`. +For comprehensive documentation on entry points, forward step functions, and customization patterns, see [entry points](/training/entry-points). #### Forward Step Customization @@ -1525,7 +1530,7 @@ pretrain(cfg, forward_step_func=custom_forward_step) - **Megatron Bridge**: Loss function returns dict with format `{key: [value, count]}` for automatic reduction - **Reduction logic**: Megatron Bridge automatically averages `value/count` across microbatches and data parallel ranks -The training loop in Megatron Bridge (see {py:func}`bridge.training.train.train_step`) automatically: +The training loop in Megatron Bridge (see `bridge.training.train.train_step`) automatically: 1. Calls the loss function for each microbatch 2. Aggregates results across microbatches 3. Performs data-parallel all-reduce @@ -1539,7 +1544,7 @@ Use custom forward steps when you need: - Additional metrics computed during training - Specialized batch preprocessing -For complete documentation on entry point signatures, loss calculation patterns, state access, and more advanced customization options, see {doc}`training/entry-points`. +For complete documentation on entry point signatures, loss calculation patterns, state access, and more advanced customization options, see [entry points](/training/entry-points). --- @@ -1629,7 +1634,7 @@ config = ConfigContainer( ) ``` -For comprehensive documentation on communication overlap strategies (TP, PP, DP, CP, MoE), hardware requirements, and performance tuning, see {doc}`training/communication-overlap`. +For comprehensive documentation on communication overlap strategies (TP, PP, DP, CP, MoE), hardware requirements, and performance tuning, see [communication overlap](/training/communication-overlap). ### Preemption Handling @@ -1653,7 +1658,7 @@ train_config = TrainingConfig( ) ``` -For more details on preemption handling and fault tolerance, see {doc}`training/resiliency`. +For more details on preemption handling and fault tolerance, see [resiliency](/training/resiliency). ### Experimental Features @@ -1744,7 +1749,7 @@ apply_deepep(model) Megatron Bridge supports both direct Python execution and NeMo-Run orchestration. While NeMo 2.0 relied heavily on NeMo-Run's recipe system, Megatron Bridge provides more flexibility. -For complete details on launching training jobs, configuration overrides, and NeMo-Run integration, see {doc}`recipe-usage`. +For complete details on launching training jobs, configuration overrides, and NeMo-Run integration, see [recipe usage](/training/recipe-usage). ### NeMo-Run Integration @@ -1790,7 +1795,7 @@ run.run(task, executor=executor) | **Plugin** | **NeMo 2.0** | **Megatron Bridge** | |------------|-------------|-----------| -| **Nsys** | `nemo.lightning.run.plugins.NsysPlugin` | {py:class}`bridge.recipes.run_plugins.NsysPlugin` | -| **Wandb** | `nemo.lightning.run.plugins.WandbPlugin` | {py:class}`bridge.recipes.run_plugins.WandbPlugin` | -| **Preemption** | `nemo.lightning.run.plugins.PreemptionPlugin` | {py:class}`bridge.recipes.run_plugins.PreemptionPlugin` | -| **Fault Tolerance** | `nemo.lightning.run.plugins.FaultTolerancePlugin` | {py:class}`bridge.recipes.run_plugins.FaultTolerancePlugin` | +| **Nsys** | `nemo.lightning.run.plugins.NsysPlugin` | `bridge.recipes.run_plugins.NsysPlugin` | +| **Wandb** | `nemo.lightning.run.plugins.WandbPlugin` | `bridge.recipes.run_plugins.WandbPlugin` | +| **Preemption** | `nemo.lightning.run.plugins.PreemptionPlugin` | `bridge.recipes.run_plugins.PreemptionPlugin` | +| **Fault Tolerance** | `nemo.lightning.run.plugins.FaultTolerancePlugin` | `bridge.recipes.run_plugins.FaultTolerancePlugin` | diff --git a/docs/get-started/prerequisites.mdx b/docs/get-started/prerequisites.mdx new file mode 100644 index 0000000000..eb85f38233 --- /dev/null +++ b/docs/get-started/prerequisites.mdx @@ -0,0 +1,32 @@ +--- +title: "Prerequisites" +description: "System, software, credential, and repository prerequisites for Megatron Bridge." +--- + +# Prerequisites + +Megatron Bridge workflows range from lightweight conversion experiments to large multi-node training jobs. Confirm the following prerequisites before installation. + +## Hardware Requirements + +- NVIDIA GPU access is recommended for conversion validation, inference, and training. +- Multi-GPU or multi-node systems are required for tensor, pipeline, context, expert, or data-parallel training at scale. +- Sufficient local or shared storage is required for Hugging Face artifacts, Megatron checkpoints, logs, and profiling outputs. + +## Software Requirements + +- Linux or a compatible container runtime environment. +- Python package tooling for installing the repository. +- Docker with NVIDIA GPU support if you use the NeMo Framework container. +- Git with submodule support for `3rdparty/Megatron-LM`. + +## Credentials + +- Hugging Face Hub access token for gated model repositories. +- Optional experiment tracking credentials for integrations such as Weights & Biases. + +## Related Topics + +- [Installation](/get-started/installation) +- [Hugging Face Conversion Quickstart](/get-started/quickstart/hugging-face-conversion) +- [Supported Models](/models) diff --git a/docs/bridge-guide.md b/docs/get-started/quickstart/hugging-face-conversion.mdx similarity index 95% rename from docs/bridge-guide.md rename to docs/get-started/quickstart/hugging-face-conversion.mdx index 93537951f6..9757468505 100644 --- a/docs/bridge-guide.md +++ b/docs/get-started/quickstart/hugging-face-conversion.mdx @@ -1,3 +1,8 @@ +--- +title: "Get Started with Hugging Face Conversion" +description: "" +--- + # Get Started with ๐Ÿค— Hugging Face Conversion Megatron Bridge provides seamless bidirectional conversion between ๐Ÿค— Hugging Face Transformers and Megatron model definitions. This guide covers the main APIs for loading models, checking compatibility, and converting between formats. @@ -5,11 +10,11 @@ Megatron Bridge provides seamless bidirectional conversion between ๐Ÿค— Hugging ## Design and Goals - Single high-level entry point: `AutoBridge` detects HF model architectures and dispatches to the correct bridge. -- Bidirectional conversion: Import HF โ†’ Megatron for training; export Megatron โ†’ HF for deployment. +- Bidirectional conversion: Import HF -> Megatron for training; export Megatron -> HF for deployment. - Parallelism-aware: Handles TP/PP/VPP/CP/EP/ETP distributions during conversion. - Streaming and memory efficiency: per-parameter streaming using safetensors. - Provider pattern: Configure Megatron-Core `TransformerConfig`-compatible attributes before instantiation via `to_megatron_provider()`. -- Convenience workflows: `import_ckpt` and `export_ckpt` provide one-call HFโ†”Megatron checkpoint flows. +- Convenience workflows: `import_ckpt` and `export_ckpt` provide one-call HF to and from Megatron checkpoint flows. See the repository `README.md` for installation, supported models, and project highlights. @@ -232,7 +237,7 @@ For more examples and advanced usage patterns, see the `examples/conversion/` di These examples can be run directly as shell commands. -### HF โ†’ Megatron checkpoint import (one call) +### HF -> Megatron checkpoint import (one call) ```bash huggingface-cli login --token @@ -246,7 +251,7 @@ huggingface-cli login --token uv run python examples/conversion/convert_checkpoints.py import --hf-model meta-llama/Llama-3.2-1B --megatron-path ./megatron_checkpoints/llama32_1b ``` -### Megatron โ†’ HF export (one call) +### Megatron -> HF export (one call) ```bash uv run python -c "from megatron.bridge import AutoBridge; b=AutoBridge.from_hf_pretrained('meta-llama/Llama-3.2-1B'); b.export_ckpt('./megatron_checkpoints/llama32_1b','./hf_exports/llama32_1b')" @@ -284,7 +289,7 @@ uv run python -m torch.distributed.run --nproc-per-node=2 -m examples.conversion ## AutoBridge API Reference -Latest public APIs and signatures (see {doc}`apidocs/bridge/bridge.models.conversion.auto_bridge`): +Latest public APIs and signatures (see [bridge.models.conversion.auto bridge](/reference/api)): ```python from megatron.bridge import AutoBridge @@ -300,10 +305,10 @@ AutoBridge.supports(config: Any) -> bool AutoBridge.to_megatron_provider(load_weights: bool = True, hf_path: str | Path | None = None) -> GPTModelProvider AutoBridge.to_megatron_model(load_weights: bool = True, hf_path: str | Path | None = None, **kwargs) -> list[MegatronModule] -# HF โ†’ Megatron weights +# HF -> Megatron weights AutoBridge.load_hf_weights(model: list[MegatronModule], hf_path: str | Path | None = None) -> None -# Megatron โ†’ HF conversion +# Megatron -> HF conversion AutoBridge.export_hf_weights(model: list[MegatronModule], cpu: bool = False, show_progress: bool = True, conversion_tasks: Optional[list[WeightConversionTask]] = None) -> Iterable[HFWeightTuple] AutoBridge.save_hf_pretrained(model: list[MegatronModule], path: str | Path, show_progress: bool = True) -> None AutoBridge.save_hf_weights(model: list[MegatronModule], path: str | Path, show_progress: bool = True) -> None @@ -313,8 +318,8 @@ AutoBridge.save_megatron_model(model: list[MegatronModule], path: str | Path) -> AutoBridge.load_megatron_model(path: str | Path, **kwargs) -> list[MegatronModule] # One-call workflows -AutoBridge.import_ckpt(hf_model_id: str | Path, megatron_path: str | Path, **kwargs) -> None # HF โ†’ Megatron ckpt -AutoBridge.export_ckpt(megatron_path: str | Path, hf_path: str | Path, show_progress: bool = True) -> None # Megatron โ†’ HF +AutoBridge.import_ckpt(hf_model_id: str | Path, megatron_path: str | Path, **kwargs) -> None # HF -> Megatron ckpt +AutoBridge.export_ckpt(megatron_path: str | Path, hf_path: str | Path, show_progress: bool = True) -> None # Megatron -> HF # Config extraction AutoBridge.transformer_config -> TransformerConfig diff --git a/docs/get-started/quickstart/index.mdx b/docs/get-started/quickstart/index.mdx new file mode 100644 index 0000000000..a763712e9c --- /dev/null +++ b/docs/get-started/quickstart/index.mdx @@ -0,0 +1,48 @@ +--- +title: "Quickstart" +description: "Quickstart paths for using Megatron Bridge for conversion and training." +--- + +# Quickstart + +Use these quickstart paths to get to a working Megatron Bridge workflow quickly. + + + + +Load a supported Hugging Face model, create a Megatron provider, and export weights. + + + +Launch model-specific recipes and override training configuration. + + + +Understand the `ConfigContainer` object used by training workflows. + + + +Choose a model-family guide before running model-specific scripts. + + + + +## Minimal Conversion Flow + +```python +from megatron.bridge import AutoBridge + +bridge = AutoBridge.from_hf_pretrained("meta-llama/Llama-3.2-1B") +provider = bridge.to_megatron_provider() +provider.tensor_model_parallel_size = 1 +provider.pipeline_model_parallel_size = 1 +provider.finalize() +model = provider.provide_distributed_model(wrap_with_ddp=False) +bridge.save_hf_pretrained(model, "./hf_exports/llama32_1b") +``` + +## Next Steps + +- [Hugging Face Conversion Quickstart](/get-started/quickstart/hugging-face-conversion) +- [Training and Customization](/training) +- [Troubleshooting](/troubleshooting) diff --git a/docs/index.md b/docs/index.md deleted file mode 100644 index 227196831a..0000000000 --- a/docs/index.md +++ /dev/null @@ -1,130 +0,0 @@ -```{include} ../README.md -:relative-docs: docs/ -``` - -```{toctree} -:caption: Performance -:hidden: - -performance-summary.md -performance-summary-archive.md -``` - -```{toctree} -:caption: Guides -:hidden: - -parallelisms.md -performance-guide.md -recipe-usage.md -nemo2-migration-guide.md -megatron-lm-to-megatron-bridge.md -``` - -```{toctree} -:caption: Bridge with ๐Ÿค— Hugging Face -:hidden: - -bridge-guide.md -bridge-tech-details.md -``` - -```{toctree} -:caption: Supported Models -:hidden: - -models/README.md -models/bailing/index.md -models/deepseek/index.md -models/falcon/index.md -models/gemma/index.md -models/glm/index.md -models/gpt_oss/index.md -models/kimi/index.md -models/llama/index.md -models/minimax/index.md -models/mistral/index.md -models/mimo/index.md -models/moonlight/index.md -models/nemotron/index.md -models/olmoe/index.md -models/qwen/index.md -models/sarvam/index.md -models/stepfun/index.md -``` - -```{toctree} -:caption: Training and Customization -:hidden: - -training/config-container-overview.md -training/entry-points.md -training/data-preparation.md -training/training-loop-settings.md -training/optimizer-scheduler.md -training/logging.md -training/profiling.md -training/memory-estimator.md -training/checkpointing.md -training/megatron-fsdp.md -training/resiliency.md -training/mixed-precision.md -training/cuda-graphs.md -training/hierarchical-context-parallel.md -training/communication-overlap.md -training/attention-optimizations.md -training/activation-recomputation.md -training/cpu-offloading.md -training/moe-optimization.md -training/peft.md -training/packed-sequences.md -training/multi-token-prediction.md -training/callbacks.md -``` - -```{toctree} -:caption: Model Optimization with ModelOpt -:hidden: - -modelopt/quantization.md -modelopt/distillation.md -modelopt/pruning.md -``` - -```{toctree} -:caption: Development -:hidden: - -adding-new-models.md -bridge-rl-integration.md -documentation.md -apidocs/index.rst -``` - -```{toctree} -:caption: Releases -:hidden: - -releases/release-process.md -releases/cherrypick-policy.md -releases/software-versions.md -releases/changelog.md -releases/known-issues.md -``` - -```{toctree} -:caption: Agent Skills -:hidden: - -skills-index -``` - -```{toctree} -:caption: Directory Readme Files -:hidden: - -README.md -releases/README.md -training/README.md -modelopt/README.md -``` diff --git a/docs/index.mdx b/docs/index.mdx new file mode 100644 index 0000000000..93a3f6f3ff --- /dev/null +++ b/docs/index.mdx @@ -0,0 +1,38 @@ +--- +title: "Home" +description: "Megatron Bridge documentation for model conversion, scalable training, model optimization, and supported model families." +--- + +# Megatron Bridge + +Megatron Bridge is a PyTorch-native library in the NVIDIA NeMo Framework for converting Hugging Face models to Megatron Core, training and fine-tuning large models, and exporting checkpoints back to Hugging Face-compatible formats. + +Use this documentation to get started with model conversion, configure distributed training, find model-family guides, and review release compatibility. + + + + +Install Megatron Bridge, review prerequisites, and run the first conversion or training workflow. + + + +Load a Hugging Face model, create a Megatron provider, and export weights back to Hugging Face format. + + + +Browse supported language, vision-language, audio, and multimodal model families. + + + +Configure training loops, recipes, checkpointing, performance tuning, PEFT, and resiliency. + + + +Apply quantization, distillation, and pruning workflows with NVIDIA ModelOpt. + + + +Find known issues, setup checks, and common runtime workarounds. + + + diff --git a/docs/modelopt/distillation.md b/docs/model-optimization/distillation.mdx similarity index 99% rename from docs/modelopt/distillation.md rename to docs/model-optimization/distillation.mdx index 163941f627..477d6d7e01 100644 --- a/docs/modelopt/distillation.md +++ b/docs/model-optimization/distillation.mdx @@ -1,3 +1,8 @@ +--- +title: "Knowledge Distillation" +description: "" +--- + # Knowledge Distillation Megatron Bridge provides a streamlined setup for Knowledge Distillation (KD) training, making it easy to enable and integrate into your workflow. This section explains how to use this feature effectively. diff --git a/docs/modelopt/README.md b/docs/model-optimization/index.mdx similarity index 63% rename from docs/modelopt/README.md rename to docs/model-optimization/index.mdx index e8f4c913ff..32cc2636e4 100644 --- a/docs/modelopt/README.md +++ b/docs/model-optimization/index.mdx @@ -1,3 +1,8 @@ +--- +title: "Model Optimization" +description: "" +--- + # Model Optimization This directory contains comprehensive documentation for optimizing models with Megatron Bridge using NVIDIA ModelOpt. Learn how to apply various optimization techniques to improve inference efficiency while maintaining model quality. @@ -15,16 +20,16 @@ NVIDIA ModelOpt provides a suite of model optimization techniques for improving ### I want to **๐Ÿ”ง Quantize a pretrained model** -โ†’ See the [Post-Training Quantization section](quantization.md#post-training-quantization-ptq) for complete PTQ workflows (quantize, resume and generate, export) +-> See the [Post-Training Quantization section](/model-optimization/quantization#post-training-quantization-ptq) for complete PTQ workflows (quantize, resume and generate, export) **๐Ÿ‹๏ธ Train with quantization** -โ†’ Check the [Quantization-Aware Training section](quantization.md#quantization-aware-training-qat) for QAT workflows +-> Check the [Quantization-Aware Training section](/model-optimization/quantization#quantization-aware-training-qat) for QAT workflows **๐ŸŽ“ Distill a teacher model into a smaller student** -โ†’ See [Knowledge Distillation](distillation.md) for the KD configuration and recipe workflow +-> See [Knowledge Distillation](/model-optimization/distillation) for the KD configuration and recipe workflow **โœ‚๏ธ Prune a model to reduce size** -โ†’ See [Pruning](pruning.md) for Minitron-based pruning to a target parameter count or fixed architecture +-> See [Pruning](/model-optimization/pruning) for Minitron-based pruning to a target parameter count or fixed architecture ## References diff --git a/docs/modelopt/pruning.md b/docs/model-optimization/pruning.mdx similarity index 96% rename from docs/modelopt/pruning.md rename to docs/model-optimization/pruning.mdx index e1ead7da9b..0c9dc8ac6a 100644 --- a/docs/modelopt/pruning.md +++ b/docs/model-optimization/pruning.mdx @@ -1,3 +1,8 @@ +--- +title: "Pruning" +description: "" +--- + # Pruning Pruning reduces model size by removing redundant parameters (e.g., shrinking hidden dimensions or layers) while preserving accuracy. In Megatron Bridge, pruning is provided by [NVIDIA Model Optimizer (ModelOpt)](https://github.com/NVIDIA/Model-Optimizer) using the Minitron algorithm for GPT and Mamba-based models loaded from HuggingFace. @@ -75,4 +80,4 @@ For more details, see the [ModelOpt pruning README](https://github.com/NVIDIA/Mo ## Next steps: Knowledge Distillation -Knowledge Distillation is required to recover the performance of the pruned model. See the [Knowledge Distillation](distillation.md) guide for more details. +Knowledge Distillation is required to recover the performance of the pruned model. See the [Knowledge Distillation](/model-optimization/distillation) guide for more details. diff --git a/docs/modelopt/quantization.md b/docs/model-optimization/quantization.mdx similarity index 98% rename from docs/modelopt/quantization.md rename to docs/model-optimization/quantization.mdx index dbffa5c971..c246122c5e 100644 --- a/docs/modelopt/quantization.md +++ b/docs/model-optimization/quantization.mdx @@ -1,3 +1,8 @@ +--- +title: "Quantization" +description: "" +--- + # Quantization This guide covers model quantization in Megatron Bridge using NVIDIA ModelOpt, including post-training quantization (PTQ) and quantization-aware training (QAT). @@ -51,7 +56,7 @@ Quantization Aware Training (QAT) helps to improve the model accuracy beyond pos ## Post-Training Quantization (PTQ) -PTQ quantizes a pretrained model by running calibration with a small dataset to compute scaling factors. The complete workflow includes: quantize โ†’ resume and generate โ†’ export. +PTQ quantizes a pretrained model by running calibration with a small dataset to compute scaling factors. The complete workflow includes: quantize -> resume and generate -> export. ### Quantize diff --git a/docs/models/README.md b/docs/models/README.md deleted file mode 100644 index 113d1369dc..0000000000 --- a/docs/models/README.md +++ /dev/null @@ -1,96 +0,0 @@ -# Supported Models - -This directory contains family-organized documentation for models supported by -Megatron Bridge. Each model page covers supported variants, Hugging Face <-> -Megatron Bridge conversion, training recipe links, and model-specific notes. - -## Family Index - -| Family | Model documentation | -|----------------|---------------------| -| **Bailing** | [Ling 2.0](bailing/ling-2.md) | -| **DeepSeek** | [DeepSeek V2](deepseek/deepseek-v2.md), [DeepSeek V3](deepseek/deepseek-v3.md), [DeepSeek V4](deepseek/deepseek-v4.md) | -| **Falcon** | [Falcon H1](falcon/falcon-h1.md) | -| **Gemma** | [Gemma](gemma/gemma.md), [Gemma 2](gemma/gemma2.md), [Gemma 3](gemma/gemma3.md), [Gemma 3 VL](gemma/gemma3-vl.md), [Gemma 4 VL](gemma/gemma4-vl.md) | -| **GLM** | [GLM 4.5](glm/glm45.md), [GLM-4.5V](glm/glm-45v.md), [GLM-4.7 / 4.7-Flash](glm/glm47.md), [GLM-5 / 5.1](glm/glm5.md) | -| **GPT-OSS** | [GPT OSS](gpt_oss/gpt-oss.md) | -| **Kimi** | [Kimi K2](kimi/kimi-k2.md), [Kimi-K2.5-VL](kimi/kimi-k25-vl.md) | -| **Llama** | [Llama 2](llama/llama2.md), [Llama 3](llama/llama3.md) | -| **MiniMax** | [MiniMax-M2 / M2.5 / M2.7](minimax/minimax-m2.md) | -| **Mistral** | [Mistral](mistral/mistral.md), [Ministral 3](mistral/ministral3.md) | -| **Xiaomi-MiMo** | [Xiaomi-MiMo](mimo/mimo.md) | -| **Moonlight** | [Moonlight](moonlight/moonlight.md) | -| **Nemotron** | [Llama Nemotron](nemotron/llama-nemotron.md), [Nemotron H and Nemotron Nano v2](nemotron/nemotronh.md), [Nemotron-3 Nano](nemotron/nemotron3-nano.md), [Nemotron-3 Super](nemotron/nemotron3-super.md), [Nemotron Nano V2 VL](nemotron/nemotron-nano-v2-vl.md), [Nemotron-3 Nano Omni](nemotron/nemotron-3-omni.md) | -| **OLMoE** | [OLMoE](olmoe/olmoe.md) | -| **Qwen** | [Qwen](qwen/qwen.md), [Qwen3-MoE](qwen/qwen3-moe.md), [Qwen3-Next](qwen/qwen3-next.md), [Qwen2.5-VL](qwen/qwen2.5-vl.md), [Qwen3-VL](qwen/qwen3-vl.md), [Qwen3.5 / 3.6](qwen/qwen35-vl.md), [Qwen2-Audio](qwen/qwen2-audio.md), [Qwen2.5-Omni](qwen/qwen25-omni.md), [Qwen3-Omni](qwen/qwen3-omni.md), [Qwen3-ASR](qwen/qwen3-asr.md) | -| **Sarvam** | [Sarvam](sarvam/sarvam.md) | -| **StepFun** | [Step-3.5-Flash](stepfun/step35.md) | - -## Quick Navigation - -### I want to - -**Find model-specific docs** --> Browse the family index above or use the navigation for the model's family. - -**Convert models between formats** --> See [Bridge Guide](../bridge-guide.md) for Hugging Face <-> Megatron -conversion basics. Model pages include model-specific commands where available. - -**Get started with training** --> See [Training Documentation](../training/README.md) for training guides and -[Recipe Usage](../recipe-usage.md) for pre-configured training recipes. - -**Add support for a new model** --> Refer to [Adding New Models](../adding-new-models.md). - -## Model Documentation Structure - -Each model documentation page typically includes: - -1. **Model Overview** - Architecture and key features -2. **Available Variants** - Supported model sizes and configurations -3. **Conversion Examples** - Converting between Hugging Face and Megatron formats -4. **Training Recipes** - Links to training configurations and examples -5. **Architecture Details** - Model-specific features and configurations - -## Model Support Overview - -### Decoder-Only and Hybrid Backbones - -- Bailing, DeepSeek, Falcon, Gemma, GLM, GPT-OSS, Kimi, Llama, MiniMax, Mistral, Moonlight, Nemotron, OLMoE, Qwen, Sarvam, StepFun, and Xiaomi-MiMo -- MoE and hybrid variants including Bailing, DeepSeek, GLM, GPT-OSS, MiniMax, Nemotron-3, OLMoE, Qwen3-MoE, Qwen3-Next, and Sarvam - -### Multimodal Variants - -- Gemma 3 VL and Gemma 4 VL -- GLM-4.5V -- Kimi-K2.5-VL -- Ministral 3 -- Nemotron Nano V2 VL and Nemotron-3 Nano Omni -- Qwen2-Audio, Qwen2.5-VL, Qwen2.5-Omni, Qwen3-VL, Qwen3.5 / 3.6, Qwen3-Omni, and Qwen3-ASR - -## Related Documentation - -- **[Main Documentation](../README.md)** - Return to main documentation -- **[Bridge Guide](../bridge-guide.md)** - Hugging Face <-> Megatron conversion -- **[Bridge Tech Details](../bridge-tech-details.md)** - Technical details of the bridge system -- **[Training Documentation](../training/README.md)** - Comprehensive training guides -- **[Adding New Models](../adding-new-models.md)** - Extending model support -- **[Recipe Usage](../recipe-usage.md)** - Using pre-configured training recipes - -## Conversion Support - -All model pages document support for one or both conversion directions: - -- **Hugging Face -> Megatron Bridge**: Load pretrained weights for training -- **Megatron Bridge -> Hugging Face**: Export trained models for deployment - -Conversion features: - -- Automatic architecture detection -- Parallelism-aware conversion (TP/PP/VPP/CP/EP) -- Streaming and memory-efficient transfers -- Verification mechanisms for conversion accuracy - -Refer to the [Bridge Guide](../bridge-guide.md) for detailed conversion instructions. diff --git a/docs/models/bailing/index.md b/docs/models/bailing/index.mdx similarity index 54% rename from docs/models/bailing/index.md rename to docs/models/bailing/index.mdx index 40ab461d98..264c1f9504 100644 --- a/docs/models/bailing/index.md +++ b/docs/models/bailing/index.mdx @@ -1,13 +1,12 @@ +--- +title: "Bailing" +description: "" +--- + # Bailing Bailing model documentation is organized by model variant. -```{toctree} -:hidden: - -ling-2.md -``` - | Variant | Guide | |---------|-------| -| Ling 2.0 | [ling-2.md](ling-2.md) | +| Ling 2.0 | [ling-2.md](/models/bailing/ling-2) | diff --git a/docs/models/bailing/ling-2.md b/docs/models/bailing/ling-2.mdx similarity index 97% rename from docs/models/bailing/ling-2.md rename to docs/models/bailing/ling-2.mdx index 8b4da495e9..0e8af45a65 100644 --- a/docs/models/bailing/ling-2.md +++ b/docs/models/bailing/ling-2.mdx @@ -1,3 +1,8 @@ +--- +title: "Ling 2.0" +description: "" +--- + # Ling 2.0 [Ling 2.0](https://github.com/inclusionAI/Ling-V2), documented in this repository under the Bailing family, is a high-sparsity Mixture-of-Experts language model family from inclusionAI. Megatron Bridge supports the Ling 2.0 MoE checkpoints through the Bailing bridge with auto-detected Hugging Face configuration and weight mapping. @@ -25,4 +30,3 @@ For checkpoint import/export, round-trip validation, and inference commands, see - Bridge implementation: [`src/megatron/bridge/models/bailing`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/src/megatron/bridge/models/bailing) - Examples: [`examples/models/bailing`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/models/bailing) - diff --git a/docs/models/deepseek/deepseek-v2.md b/docs/models/deepseek/deepseek-v2.mdx similarity index 94% rename from docs/models/deepseek/deepseek-v2.md rename to docs/models/deepseek/deepseek-v2.mdx index 34651b90b5..c1631d8dc1 100644 --- a/docs/models/deepseek/deepseek-v2.md +++ b/docs/models/deepseek/deepseek-v2.mdx @@ -1,3 +1,8 @@ +--- +title: "DeepSeek V2" +description: "" +--- + # DeepSeek V2 [DeepSeek-V2](https://github.com/deepseek-ai/DeepSeek-V2) is a Mixture-of-Experts (MoE) language model that uses innovative Multi-head Latent Attention (MLA) for efficient inference and DeepSeekMoE architecture for economical training and inference. The model achieves performance comparable to GPT-4 while using significantly fewer activated parameters. More information is available in the companion paper ["DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model"](https://arxiv.org/abs/2405.04434). @@ -24,7 +29,7 @@ Both models support pretraining with expert parallelism for efficient MoE traini ## Conversion with ๐Ÿค— Hugging Face -### Load HF โ†’ Megatron +### Load HF -> Megatron ```python from megatron.bridge import AutoBridge @@ -50,7 +55,7 @@ uv run python examples/conversion/convert_checkpoints.py import \ --trust-remote-code ``` -### Export Megatron โ†’ HF +### Export Megatron -> HF ```python from megatron.bridge import AutoBridge @@ -81,7 +86,7 @@ For more details, see [examples/conversion/hf_to_megatron_generate_text.py](http ## Recipes -See: [bridge.recipes.deepseek.deepseek_v2](../../apidocs/bridge/bridge.recipes.deepseek.deepseek_v2.md) +See: [bridge.recipes.deepseek.deepseek_v2](/reference/api) ### Available Recipes @@ -155,7 +160,6 @@ Finetuning recipes for DeepSeek V2 models are not currently available. - GitHub Repository: https://github.com/deepseek-ai/DeepSeek-V2 ## Related Docs -- Recipe usage: [Recipe usage](../../recipe-usage.md) -- Customizing the training recipe configuration: [Configuration overview](../../training/config-container-overview.md) -- Training entry points: [Entry points](../../training/entry-points.md) - +- Recipe usage: [Recipe usage](/training/recipe-usage) +- Customizing the training recipe configuration: [Configuration overview](/training/config-container-overview) +- Training entry points: [Entry points](/training/entry-points) diff --git a/docs/models/deepseek/deepseek-v3.md b/docs/models/deepseek/deepseek-v3.mdx similarity index 94% rename from docs/models/deepseek/deepseek-v3.md rename to docs/models/deepseek/deepseek-v3.mdx index d29544600f..1f0d6306a0 100644 --- a/docs/models/deepseek/deepseek-v3.md +++ b/docs/models/deepseek/deepseek-v3.mdx @@ -1,3 +1,8 @@ +--- +title: "DeepSeek V3" +description: "" +--- + # DeepSeek V3 [DeepSeek-V3](https://github.com/deepseek-ai/DeepSeek-V3) is a large-scale Mixture-of-Experts (MoE) language model with 671B total parameters and 37B activated parameters per token. It features Multi-head Latent Attention (MLA), innovative load balancing strategies, and Multi-Token Prediction (MTP) for improved training efficiency. DeepSeek-V3 achieves state-of-the-art performance while maintaining economical training costs. More information is available in the technical report ["DeepSeek-V3 Technical Report"](https://arxiv.org/abs/2412.19437). @@ -25,7 +30,7 @@ The model supports pretraining with expert parallelism, pipeline parallelism, an ## Conversion with ๐Ÿค— Hugging Face -### Load HF โ†’ Megatron +### Load HF -> Megatron ```python from megatron.bridge import AutoBridge @@ -51,7 +56,7 @@ uv run python examples/conversion/convert_checkpoints.py import \ --trust-remote-code ``` -### Export Megatron โ†’ HF +### Export Megatron -> HF ```python from megatron.bridge import AutoBridge @@ -84,7 +89,7 @@ For more details, see [examples/conversion/hf_to_megatron_generate_text.py](http ## Recipes -See: [bridge.recipes.deepseek.deepseek_v3](../../apidocs/bridge/bridge.recipes.deepseek.deepseek_v3.md) +See: [bridge.recipes.deepseek.deepseek_v3](/reference/api) ### Available Recipes @@ -154,6 +159,6 @@ Finetuning recipes for DeepSeek V3 are not currently available. - GitHub Repository: https://github.com/deepseek-ai/DeepSeek-V3 ## Related Docs -- Recipe usage: [Recipe usage](../../recipe-usage.md) -- Customizing the training recipe configuration: [Configuration overview](../../training/config-container-overview.md) -- Training entry points: [Entry points](../../training/entry-points.md) +- Recipe usage: [Recipe usage](/training/recipe-usage) +- Customizing the training recipe configuration: [Configuration overview](/training/config-container-overview) +- Training entry points: [Entry points](/training/entry-points) diff --git a/docs/models/deepseek/deepseek-v4.md b/docs/models/deepseek/deepseek-v4.mdx similarity index 95% rename from docs/models/deepseek/deepseek-v4.md rename to docs/models/deepseek/deepseek-v4.mdx index 88b7dd8676..ae4c819c3c 100644 --- a/docs/models/deepseek/deepseek-v4.md +++ b/docs/models/deepseek/deepseek-v4.mdx @@ -1,3 +1,8 @@ +--- +title: "DeepSeek V4" +description: "" +--- + # DeepSeek V4 [DeepSeek-V4](https://github.com/deepseek-ai/DeepSeek-V4) is the next-generation Mixture-of-Experts language model from DeepSeek-AI. It extends the V3 design with **Hyper-Connections (mHC)** for multi-stream residual mixing, **Compressed Sparse Attention (CSA)** with a learned token-importance indexer (DSA), **hash-routed MoE layers** for the first few decoder blocks, and a refined **Multi-Token Prediction (MTP)** head with separate `e_proj` / `h_proj` projections. @@ -9,7 +14,7 @@ DeepSeek V4 models are supported via the Bridge system with auto-detected config - **Hybrid Attention (DSv4HybridSelfAttention)**: Per-layer mix of dense MLA and Compressed Sparse Attention selected by `compress_ratios` - **Compressed Sparse Attention (CSA)** with **DSA Indexer**: Top-k token selection over windowed keys; `index_n_heads`, `index_head_dim`, `index_topk` control the indexer - **Hyper-Connections (mHC)**: 4-stream residual mixing per layer (`hc_mult = 4`) with sinkhorn-iterated attention; per-MTP-layer `hc_head_*` learns output contraction -- **Hash-Routed MoE**: First few decoder layers use a deterministic vocab โ†’ expert mapping (`tid2eid`) instead of softmax routing +- **Hash-Routed MoE**: First few decoder layers use a deterministic vocab -> expert mapping (`tid2eid`) instead of softmax routing - **Multi-Token Prediction (MTP)**: One MTP layer with separate `e_proj` and `h_proj` projections (post-MCore #4518) - **YaRN RoPE**: `rotary_scaling_factor=16`, `original_max_position_embeddings=65536`; `mscale=mscale_all_dim=1.0` for V4 - **Sigmoid Gating with Expert Bias**: `noaux_tc` load balancing, `sqrtsoftplus` scoring, expert bias enabled diff --git a/docs/models/deepseek/index.md b/docs/models/deepseek/index.md deleted file mode 100644 index 5bd6f856bc..0000000000 --- a/docs/models/deepseek/index.md +++ /dev/null @@ -1,17 +0,0 @@ -# DeepSeek - -DeepSeek model documentation is organized by model variant. - -```{toctree} -:hidden: - -deepseek-v2.md -deepseek-v3.md -deepseek-v4.md -``` - -| Variant | Guide | -|---------|-------| -| DeepSeek V2 | [deepseek-v2.md](deepseek-v2.md) | -| DeepSeek V3 | [deepseek-v3.md](deepseek-v3.md) | -| DeepSeek V4 | [deepseek-v4.md](deepseek-v4.md) | diff --git a/docs/models/deepseek/index.mdx b/docs/models/deepseek/index.mdx new file mode 100644 index 0000000000..509cc613c0 --- /dev/null +++ b/docs/models/deepseek/index.mdx @@ -0,0 +1,14 @@ +--- +title: "DeepSeek" +description: "" +--- + +# DeepSeek + +DeepSeek model documentation is organized by model variant. + +| Variant | Guide | +|---------|-------| +| DeepSeek V2 | [deepseek-v2.md](/models/deepseek/deepseek-v2) | +| DeepSeek V3 | [deepseek-v3.md](/models/deepseek/deepseek-v3) | +| DeepSeek V4 | [deepseek-v4.md](/models/deepseek/deepseek-v4) | diff --git a/docs/models/falcon/falcon-h1.md b/docs/models/falcon/falcon-h1.mdx similarity index 97% rename from docs/models/falcon/falcon-h1.md rename to docs/models/falcon/falcon-h1.mdx index 0b2f91ba23..953bbd56de 100644 --- a/docs/models/falcon/falcon-h1.md +++ b/docs/models/falcon/falcon-h1.mdx @@ -1,3 +1,8 @@ +--- +title: "Falcon H1" +description: "" +--- + # Falcon H1 [Falcon H1](https://huggingface.co/tiiuae/Falcon-H1-0.5B-Instruct) is a hybrid language model family that combines Mamba, attention, and MLP blocks in each decoder layer. Megatron Bridge supports Falcon H1 through a custom provider and model implementation. @@ -25,4 +30,3 @@ For checkpoint conversion, tokenizer asset export, round-trip validation, and in - Bridge implementation: [`src/megatron/bridge/models/falcon_h1`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/src/megatron/bridge/models/falcon_h1) - Examples: [`examples/models/falcon_h1`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/models/falcon_h1) - diff --git a/docs/models/falcon/index.md b/docs/models/falcon/index.mdx similarity index 53% rename from docs/models/falcon/index.md rename to docs/models/falcon/index.mdx index ac9cc3c213..b18de2916c 100644 --- a/docs/models/falcon/index.md +++ b/docs/models/falcon/index.mdx @@ -1,13 +1,12 @@ +--- +title: "Falcon" +description: "" +--- + # Falcon Falcon model documentation is organized by model variant. -```{toctree} -:hidden: - -falcon-h1.md -``` - | Variant | Guide | |---------|-------| -| Falcon H1 | [falcon-h1.md](falcon-h1.md) | +| Falcon H1 | [falcon-h1.md](/models/falcon/falcon-h1) | diff --git a/docs/models/gemma/gemma.md b/docs/models/gemma/gemma.mdx similarity index 97% rename from docs/models/gemma/gemma.md rename to docs/models/gemma/gemma.mdx index 53c8c4d49b..65dac6a9f3 100644 --- a/docs/models/gemma/gemma.md +++ b/docs/models/gemma/gemma.mdx @@ -1,3 +1,8 @@ +--- +title: "Gemma" +description: "" +--- + # Gemma [Gemma](https://huggingface.co/collections/google/gemma-release-65d5efbccdbb8c4202ec078b) is Google's original lightweight open model family. Megatron Bridge supports Gemma causal language models through the `GemmaBridge` implementation for the Hugging Face `GemmaForCausalLM` architecture. @@ -38,4 +43,3 @@ uv run python examples/conversion/hf_to_megatron_generate_text.py \ - Bridge implementation: [`src/megatron/bridge/models/gemma/gemma_bridge.py`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/models/gemma/gemma_bridge.py) - Conversion examples: [`examples/conversion`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/conversion) - diff --git a/docs/models/gemma/gemma2.md b/docs/models/gemma/gemma2.mdx similarity index 94% rename from docs/models/gemma/gemma2.md rename to docs/models/gemma/gemma2.mdx index 252741af65..5887eb77fe 100644 --- a/docs/models/gemma/gemma2.md +++ b/docs/models/gemma/gemma2.mdx @@ -1,3 +1,8 @@ +--- +title: "Gemma 2" +description: "" +--- + # Gemma 2 [Google's Gemma 2](https://huggingface.co/collections/google/gemma-2-release-667d6600fd5220e7b967f315) is a family of lightweight, open models built on the same research and technology used to create Gemini models. The Gemma 2 architecture builds on the transformer decoder framework with enhancements including pre-normalization with RMSNorm, GeGLU activations, Rotary Positional Embeddings (RoPE), attention logit softcapping, and sliding window attention. @@ -43,7 +48,7 @@ Gemma 2 introduces several architectural innovations: ## Conversion with ๐Ÿค— Hugging Face -### Load HF โ†’ Megatron +### Load HF -> Megatron ```python from megatron.bridge import AutoBridge @@ -59,7 +64,7 @@ provider.finalize() model = provider.provide_distributed_model(wrap_with_ddp=False) ``` -### Import HF โ†’ Megatron +### Import HF -> Megatron To import the HF model to your desired Megatron path: ```bash # Gemma 2 2B @@ -78,7 +83,7 @@ uv run python examples/conversion/convert_checkpoints.py import \ --megatron-path /models/gemma-2-27b ``` -### Export Megatron โ†’ HF +### Export Megatron -> HF ```bash # Gemma 2 9B example uv run python examples/conversion/convert_checkpoints.py export \ @@ -102,7 +107,7 @@ Note: ## Pretrain and Finetune Recipes -- See: [bridge.recipes.gemma](../../apidocs/bridge/bridge.recipes.gemma.md) +- See: [bridge.recipes.gemma](/reference/api) - Available recipes: - **Pretraining:** - `gemma2_2b_pretrain_config`: Pre-training configuration for Gemma 2 2B @@ -274,7 +279,7 @@ config = gemma2_27b_peft_config( ## Examples - Checkpoint import/export: [examples/conversion/convert_checkpoints.py](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/conversion/convert_checkpoints.py) -- Generate text (HFโ†’Megatron): [examples/conversion/hf_to_megatron_generate_text.py](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/conversion/hf_to_megatron_generate_text.py) +- Generate text (HF->Megatron): [examples/conversion/hf_to_megatron_generate_text.py](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/conversion/hf_to_megatron_generate_text.py) ## Hugging Face Model Cards @@ -284,7 +289,6 @@ config = gemma2_27b_peft_config( - Gemma 2 Collection: https://huggingface.co/collections/google/gemma-2-release-667d6600fd5220e7b967f315 ## Related Docs -- Recipe usage: [Recipe usage](../../recipe-usage.md) -- Customizing the training recipe configuration: [Configuration overview](../../training/config-container-overview.md) -- Training entry points: [Entry points](../../training/entry-points.md) - +- Recipe usage: [Recipe usage](/training/recipe-usage) +- Customizing the training recipe configuration: [Configuration overview](/training/config-container-overview) +- Training entry points: [Entry points](/training/entry-points) diff --git a/docs/models/gemma/gemma3-vl.md b/docs/models/gemma/gemma3-vl.mdx similarity index 91% rename from docs/models/gemma/gemma3-vl.md rename to docs/models/gemma/gemma3-vl.mdx index 68f3ed3ef9..752cdc37f0 100644 --- a/docs/models/gemma/gemma3-vl.md +++ b/docs/models/gemma/gemma3-vl.mdx @@ -1,3 +1,8 @@ +--- +title: "Gemma 3 VL (Vision-Language)" +description: "" +--- + # Gemma 3 VL (Vision-Language) [Google's Gemma 3 VL](https://huggingface.co/collections/google/gemma-3-release) is a family of vision-language models built on the same research and technology used to create Gemini models. The Gemma 3 VL architecture combines the text-generation capabilities of Gemma 3 with a SigLIP vision encoder for robust visual understanding. @@ -55,7 +60,7 @@ For checkpoint conversion, inference, finetuning recipes, and step-by-step train - Gemma 3 VL 27B: https://huggingface.co/google/gemma-3-27b-it ## Related Docs -- Related Gemma variant: [Gemma 3](gemma3.md) -- Recipe usage: [Recipe usage](../../recipe-usage.md) -- Customizing the training recipe configuration: [Configuration overview](../../training/config-container-overview.md) -- Training entry points: [Entry points](../../training/entry-points.md) +- Related Gemma variant: [Gemma 3](/models/gemma/gemma3) +- Recipe usage: [Recipe usage](/training/recipe-usage) +- Customizing the training recipe configuration: [Configuration overview](/training/config-container-overview) +- Training entry points: [Entry points](/training/entry-points) diff --git a/docs/models/gemma/gemma3.md b/docs/models/gemma/gemma3.mdx similarity index 91% rename from docs/models/gemma/gemma3.md rename to docs/models/gemma/gemma3.mdx index c69d0c4588..3c8e6f16ff 100644 --- a/docs/models/gemma/gemma3.md +++ b/docs/models/gemma/gemma3.mdx @@ -1,3 +1,8 @@ +--- +title: "Gemma 3" +description: "" +--- + # Gemma 3 [Google's Gemma 3](https://huggingface.co/collections/google/gemma-3-release) is a family of lightweight, state-of-the-art open models built on the same research and technology used to create Gemini models. The Gemma 3 architecture builds on the transformer decoder framework with enhancements including pre-normalization with RMSNorm, GeGLU activations, Rotary Positional Embeddings (RoPE), and hybrid attention patterns (sliding window and global attention). @@ -30,7 +35,7 @@ Gemma 3 introduces several architectural innovations: ## Conversion with ๐Ÿค— Hugging Face -### Load HF โ†’ Megatron +### Load HF -> Megatron ```python from megatron.bridge import AutoBridge @@ -46,7 +51,7 @@ provider.finalize() model = provider.provide_distributed_model(wrap_with_ddp=False) ``` -### Import HF โ†’ Megatron +### Import HF -> Megatron To import the HF model to your desired Megatron path: ```bash uv run python examples/conversion/convert_checkpoints.py import \ @@ -54,7 +59,7 @@ uv run python examples/conversion/convert_checkpoints.py import \ --megatron-path /models/gemma-3-1b-it ``` -### Export Megatron โ†’ HF +### Export Megatron -> HF ```bash uv run python examples/conversion/convert_checkpoints.py export \ --hf-model google/gemma-3-1b-it \ @@ -77,7 +82,7 @@ Note: ## Recipes -See: [bridge.recipes.gemma](../../apidocs/bridge/bridge.recipes.gemma.md) +See: [bridge.recipes.gemma](/reference/api) ### Available Recipes @@ -175,7 +180,7 @@ uv run python -m torch.distributed.run --nproc-per-node=8 run/run_recipe.py \ ## Examples - Checkpoint import/export: [examples/conversion/convert_checkpoints.py](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/conversion/convert_checkpoints.py) -- Generate text (HFโ†’Megatron): [examples/conversion/hf_to_megatron_generate_text.py](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/conversion/hf_to_megatron_generate_text.py) +- Generate text (HF->Megatron): [examples/conversion/hf_to_megatron_generate_text.py](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/conversion/hf_to_megatron_generate_text.py) ## Hugging Face Model Cards @@ -183,6 +188,6 @@ uv run python -m torch.distributed.run --nproc-per-node=8 run/run_recipe.py \ ## Related Docs - Gemma3 Vision-Language Models: [Gemma 3 VL](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/models/gemma/gemma3_vl/README.md) -- Recipe usage: [Recipe usage](../../recipe-usage.md) -- Customizing the training recipe configuration: [Configuration overview](../../training/config-container-overview.md) -- Training entry points: [Entry points](../../training/entry-points.md) +- Recipe usage: [Recipe usage](/training/recipe-usage) +- Customizing the training recipe configuration: [Configuration overview](/training/config-container-overview) +- Training entry points: [Entry points](/training/entry-points) diff --git a/docs/models/gemma/gemma4-vl.md b/docs/models/gemma/gemma4-vl.mdx similarity index 83% rename from docs/models/gemma/gemma4-vl.md rename to docs/models/gemma/gemma4-vl.mdx index ffd8fc5c01..068d4bd28e 100644 --- a/docs/models/gemma/gemma4-vl.md +++ b/docs/models/gemma/gemma4-vl.mdx @@ -1,3 +1,8 @@ +--- +title: "Gemma 4 VL (26B-A4B MoE)" +description: "" +--- + # Gemma 4 VL (26B-A4B MoE) [Google's Gemma 4 26B-A4B](https://huggingface.co/google/gemma-4-26B-A4B-it) @@ -6,7 +11,7 @@ parameters). It pairs a 128-expert top-k=8 MoE language backbone with a SigLIP vision tower, dual sliding/global attention, and K=V tying on the full-attention layers. -NeMo Megatron Bridge supports HFโ†”Megatron conversion, full SFT, and LoRA +NeMo Megatron Bridge supports HF to and from Megatron conversion, full SFT, and LoRA PEFT on image-text datasets. The finetuned model can be re-exported to ๐Ÿค— Hugging Face format for downstream evaluation or deployment. diff --git a/docs/models/gemma/index.md b/docs/models/gemma/index.md deleted file mode 100644 index 84394d88bf..0000000000 --- a/docs/models/gemma/index.md +++ /dev/null @@ -1,21 +0,0 @@ -# Gemma - -Gemma model documentation is organized by model variant. - -```{toctree} -:hidden: - -gemma.md -gemma2.md -gemma3.md -gemma3-vl.md -gemma4-vl.md -``` - -| Variant | Guide | -|---------|-------| -| Gemma | [gemma.md](gemma.md) | -| Gemma 2 | [gemma2.md](gemma2.md) | -| Gemma 3 | [gemma3.md](gemma3.md) | -| Gemma 3 VL | [gemma3-vl.md](gemma3-vl.md) | -| Gemma 4 VL | [gemma4-vl.md](gemma4-vl.md) | diff --git a/docs/models/gemma/index.mdx b/docs/models/gemma/index.mdx new file mode 100644 index 0000000000..30c937b2bf --- /dev/null +++ b/docs/models/gemma/index.mdx @@ -0,0 +1,16 @@ +--- +title: "Gemma" +description: "" +--- + +# Gemma + +Gemma model documentation is organized by model variant. + +| Variant | Guide | +|---------|-------| +| Gemma | [gemma.md](/models/gemma/gemma) | +| Gemma 2 | [gemma2.md](/models/gemma/gemma2) | +| Gemma 3 | [gemma3.md](/models/gemma/gemma3) | +| Gemma 3 VL | [gemma3-vl.md](/models/gemma/gemma3-vl) | +| Gemma 4 VL | [gemma4-vl.md](/models/gemma/gemma4-vl) | diff --git a/docs/models/glm/glm-45v.md b/docs/models/glm/glm-45v.mdx similarity index 89% rename from docs/models/glm/glm-45v.md rename to docs/models/glm/glm-45v.mdx index ad14c4dc81..dde3d59f70 100644 --- a/docs/models/glm/glm-45v.md +++ b/docs/models/glm/glm-45v.mdx @@ -1,3 +1,8 @@ +--- +title: "GLM-4.5V" +description: "" +--- + # GLM-4.5V [GLM-4.5V](https://huggingface.co/zai-org/GLM-4.5V) is a powerful vision-language model built on the GLM-4.5 Air architecture. GLM-4.5V combines a 106B parameter sparse MoE language model with a vision encoder for robust multimodal understanding of images and videos. @@ -6,9 +11,9 @@ GLM-4.5V supports multimodal tasks including image captioning, visual question a GLM family models are supported via the Bridge system with auto-detected configuration and weight mapping. -```{important} + Please update `transformers` version to 4.57.1 or higher in order to use the GLM-4.5V model. -``` + ## Available Models @@ -48,7 +53,7 @@ For checkpoint conversion, inference, finetuning recipes, and step-by-step train - GLM-4.5V: https://huggingface.co/zai-org/GLM-4.5V ## Related Docs -- Related GLM variant: [GLM 4.5](glm45.md) -- Recipe usage: [Recipe usage](../../recipe-usage.md) -- Customizing the training recipe configuration: [Configuration overview](../../training/config-container-overview.md) -- Training entry points: [Entry points](../../training/entry-points.md) +- Related GLM variant: [GLM 4.5](/models/glm/glm45) +- Recipe usage: [Recipe usage](/training/recipe-usage) +- Customizing the training recipe configuration: [Configuration overview](/training/config-container-overview) +- Training entry points: [Entry points](/training/entry-points) diff --git a/docs/models/glm/glm45.md b/docs/models/glm/glm45.mdx similarity index 95% rename from docs/models/glm/glm45.md rename to docs/models/glm/glm45.mdx index 26ee1eb5d8..d4e2940209 100644 --- a/docs/models/glm/glm45.md +++ b/docs/models/glm/glm45.mdx @@ -1,3 +1,8 @@ +--- +title: "GLM 4.5" +description: "" +--- + # GLM 4.5 [GLM 4.5](https://huggingface.co/zai-org/GLM-4.5) is a family of large-scale Mixture-of-Experts (MoE) language models developed by Zhipu AI. Built on the GLM (General Language Model) architecture, GLM 4.5 introduces advanced features including sparse MoE layers, shared expert mechanisms, and Multi-Token Prediction (MTP) for improved training efficiency and performance. @@ -62,7 +67,7 @@ GLM 4.5 introduces several advanced architectural innovations: ## Conversion with ๐Ÿค— Hugging Face -### Load HF โ†’ Megatron +### Load HF -> Megatron ```python from megatron.bridge import AutoBridge @@ -81,7 +86,7 @@ provider.finalize() model = provider.provide_distributed_model(wrap_with_ddp=False) ``` -### Import HF โ†’ Megatron +### Import HF -> Megatron ```bash # Import GLM 4.5 Air model uv run python examples/conversion/convert_checkpoints.py import \ @@ -94,7 +99,7 @@ uv run python examples/conversion/convert_checkpoints.py import \ --megatron-path /models/glm45-355b ``` -### Export Megatron โ†’ HF +### Export Megatron -> HF ```bash uv run python examples/conversion/convert_checkpoints.py export \ --hf-model zai-org/GLM-4.5-Air \ @@ -272,7 +277,7 @@ config = glm45_air_106b_pretrain_config( ## Examples - Checkpoint import/export: [examples/conversion/convert_checkpoints.py](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/conversion/convert_checkpoints.py) -- Generate text (HFโ†’Megatron): [examples/conversion/hf_to_megatron_generate_text.py](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/conversion/hf_to_megatron_generate_text.py) +- Generate text (HF->Megatron): [examples/conversion/hf_to_megatron_generate_text.py](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/conversion/hf_to_megatron_generate_text.py) ## Hugging Face Model Cards - GLM 4.5 355B: https://huggingface.co/zai-org/GLM-4.5 @@ -288,7 +293,6 @@ config = glm45_air_106b_pretrain_config( 6. **Load Balancing**: Shared expert overlap and auxiliary loss for optimal expert utilization ## Related Docs -- Recipe usage: [Recipe usage](../../recipe-usage.md) -- Customizing the training recipe configuration: [Configuration overview](../../training/config-container-overview.md) -- Training entry points: [Entry points](../../training/entry-points.md) - +- Recipe usage: [Recipe usage](/training/recipe-usage) +- Customizing the training recipe configuration: [Configuration overview](/training/config-container-overview) +- Training entry points: [Entry points](/training/entry-points) diff --git a/docs/models/glm/glm47.md b/docs/models/glm/glm47.mdx similarity index 97% rename from docs/models/glm/glm47.md rename to docs/models/glm/glm47.mdx index c55202c4c5..b73bf0ddf1 100644 --- a/docs/models/glm/glm47.md +++ b/docs/models/glm/glm47.mdx @@ -1,3 +1,8 @@ +--- +title: "GLM-4.7 and GLM-4.7-Flash" +description: "" +--- + # GLM-4.7 and GLM-4.7-Flash [GLM-4.7](https://huggingface.co/zai-org/GLM-4.7) and [GLM-4.7-Flash](https://huggingface.co/zai-org/GLM-4.7-Flash) are GLM MoE language models from Zhipu AI. Megatron Bridge supports both checkpoints, with the full GLM-4.7 model handled by the GLM-4.5 bridge registration and the Flash variant handled by a dedicated GLM-4.7-Flash bridge. @@ -25,4 +30,3 @@ For checkpoint conversion, inference, hardware notes, and validated parallelism - GLM-4.7 bridge path: [`src/megatron/bridge/models/glm/glm45_bridge.py`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/models/glm/glm45_bridge.py) - GLM-4.7-Flash bridge path: [`src/megatron/bridge/models/glm/glm47_flash_bridge.py`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/models/glm/glm47_flash_bridge.py) - Examples: [`examples/models/glm47`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/models/glm47) - diff --git a/docs/models/glm/glm5.md b/docs/models/glm/glm5.mdx similarity index 96% rename from docs/models/glm/glm5.md rename to docs/models/glm/glm5.mdx index 53deb7a56a..3722673f1c 100644 --- a/docs/models/glm/glm5.md +++ b/docs/models/glm/glm5.mdx @@ -1,3 +1,8 @@ +--- +title: "GLM-5 and GLM-5.1" +description: "" +--- + # GLM-5 and GLM-5.1 [GLM-5](https://huggingface.co/zai-org/GLM-5) and [GLM-5.1](https://huggingface.co/zai-org/GLM-5.1) are large sparse MoE language models with Multi-Latent Attention and Dynamic Sparse Attention. Megatron Bridge supports both checkpoints through the shared `GLM5Bridge`. @@ -26,4 +31,3 @@ For conversion, inference, dependency notes, hardware requirements, and MCore pa - Bridge implementation: [`src/megatron/bridge/models/glm_moe_dsa/glm5_bridge.py`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/models/glm_moe_dsa/glm5_bridge.py) - Examples: [`examples/models/glm/glm5`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/models/glm/glm5) - diff --git a/docs/models/glm/index.md b/docs/models/glm/index.md deleted file mode 100644 index b1875ee200..0000000000 --- a/docs/models/glm/index.md +++ /dev/null @@ -1,19 +0,0 @@ -# GLM - -GLM model documentation is organized by model variant. - -```{toctree} -:hidden: - -glm45.md -glm-45v.md -glm47.md -glm5.md -``` - -| Variant | Guide | -|---------|-------| -| GLM 4.5 | [glm45.md](glm45.md) | -| GLM-4.5V | [glm-45v.md](glm-45v.md) | -| GLM-4.7 / 4.7-Flash | [glm47.md](glm47.md) | -| GLM-5 / 5.1 | [glm5.md](glm5.md) | diff --git a/docs/models/glm/index.mdx b/docs/models/glm/index.mdx new file mode 100644 index 0000000000..d939596c07 --- /dev/null +++ b/docs/models/glm/index.mdx @@ -0,0 +1,15 @@ +--- +title: "GLM" +description: "" +--- + +# GLM + +GLM model documentation is organized by model variant. + +| Variant | Guide | +|---------|-------| +| GLM 4.5 | [glm45.md](/models/glm/glm45) | +| GLM-4.5V | [glm-45v.md](/models/glm/glm-45v) | +| GLM-4.7 / 4.7-Flash | [glm47.md](/models/glm/glm47) | +| GLM-5 / 5.1 | [glm5.md](/models/glm/glm5) | diff --git a/docs/models/gpt_oss/gpt-oss.md b/docs/models/gpt-oss/gpt-oss.mdx similarity index 86% rename from docs/models/gpt_oss/gpt-oss.md rename to docs/models/gpt-oss/gpt-oss.mdx index f5edba4c06..ed17707dd8 100644 --- a/docs/models/gpt_oss/gpt-oss.md +++ b/docs/models/gpt-oss/gpt-oss.mdx @@ -1,3 +1,8 @@ +--- +title: "GPT OSS" +description: "" +--- + # GPT OSS GPT OSS is a Mixture-of-Experts (MoE) language model family featuring two variants: **GPT OSS 20B** and **GPT OSS 120B**. These models are designed with advanced attention mechanisms and MoE architectures optimized for long-context understanding. @@ -52,8 +57,8 @@ For checkpoint conversion, inference, finetuning recipes, and step-by-step train ## API reference -- GPT OSS recipes: [bridge.recipes.gpt_oss](../../apidocs/bridge/bridge.recipes.gpt_oss.md) -- GPT OSS model provider: [bridge.models.gpt_oss.GPTOSSProvider](../../apidocs/bridge/bridge.models.gpt_oss.md) +- GPT OSS recipes: [bridge.recipes.gpt_oss](/reference/api) +- GPT OSS model provider: [bridge.models.gpt_oss.GPTOSSProvider](/reference/api) ## Hugging Face model cards @@ -65,7 +70,7 @@ For checkpoint conversion, inference, finetuning recipes, and step-by-step train ## Related docs -- Recipe usage and customization: [Recipe usage](../../recipe-usage.md) -- Training configuration: [Configuration overview](../../training/config-container-overview.md) -- Training entry points: [Entry points](../../training/entry-points.md) -- Attention optimizations: [Attention optimizations](../../training/attention-optimizations.md) +- Recipe usage and customization: [Recipe usage](/training/recipe-usage) +- Training configuration: [Configuration overview](/training/config-container-overview) +- Training entry points: [Entry points](/training/entry-points) +- Attention optimizations: [Attention optimizations](/training/attention-optimizations) diff --git a/docs/models/gpt_oss/index.md b/docs/models/gpt-oss/index.mdx similarity index 54% rename from docs/models/gpt_oss/index.md rename to docs/models/gpt-oss/index.mdx index 585900c820..41c3799115 100644 --- a/docs/models/gpt_oss/index.md +++ b/docs/models/gpt-oss/index.mdx @@ -1,13 +1,12 @@ +--- +title: "GPT-OSS" +description: "" +--- + # GPT-OSS GPT-OSS model documentation is organized by model variant. -```{toctree} -:hidden: - -gpt-oss.md -``` - | Variant | Guide | |---------|-------| -| GPT OSS | [gpt-oss.md](gpt-oss.md) | +| GPT OSS | [gpt-oss.md](/models/gpt-oss/gpt-oss) | diff --git a/docs/models/index.mdx b/docs/models/index.mdx new file mode 100644 index 0000000000..1bc5b45282 --- /dev/null +++ b/docs/models/index.mdx @@ -0,0 +1,101 @@ +--- +title: "Supported Models" +description: "" +--- + +# Supported Models + +This directory contains family-organized documentation for models supported by +Megatron Bridge. Each model page covers supported variants, Hugging Face to and from +Megatron Bridge conversion, training recipe links, and model-specific notes. + +## Family Index + +| Family | Model documentation | +|----------------|---------------------| +| **Bailing** | [Ling 2.0](/models/bailing/ling-2) | +| **DeepSeek** | [DeepSeek V2](/models/deepseek/deepseek-v2), [DeepSeek V3](/models/deepseek/deepseek-v3), [DeepSeek V4](/models/deepseek/deepseek-v4) | +| **Falcon** | [Falcon H1](/models/falcon/falcon-h1) | +| **Gemma** | [Gemma](/models/gemma/gemma), [Gemma 2](/models/gemma/gemma2), [Gemma 3](/models/gemma/gemma3), [Gemma 3 VL](/models/gemma/gemma3-vl), [Gemma 4 VL](/models/gemma/gemma4-vl) | +| **GLM** | [GLM 4.5](/models/glm/glm45), [GLM-4.5V](/models/glm/glm-45v), [GLM-4.7 / 4.7-Flash](/models/glm/glm47), [GLM-5 / 5.1](/models/glm/glm5) | +| **GPT-OSS** | [GPT OSS](/models/gpt-oss/gpt-oss) | +| **Kimi** | [Kimi K2](/models/kimi/kimi-k2), [Kimi-K2.5-VL](/models/kimi/kimi-k25-vl) | +| **Llama** | [Llama 2](/models/llama/llama2), [Llama 3](/models/llama/llama3) | +| **MiniMax** | [MiniMax-M2 / M2.5 / M2.7](/models/minimax/minimax-m2) | +| **Mistral** | [Mistral](/models/mistral/mistral), [Ministral 3](/models/mistral/ministral3) | +| **Xiaomi-MiMo** | [Xiaomi-MiMo](/models/mimo/mimo) | +| **Moonlight** | [Moonlight](/models/moonlight/moonlight) | +| **Nemotron** | [Llama Nemotron](/models/nemotron/llama-nemotron), [Nemotron H and Nemotron Nano v2](/models/nemotron/nemotronh), [Nemotron-3 Nano](/models/nemotron/nemotron3-nano), [Nemotron-3 Super](/models/nemotron/nemotron3-super), [Nemotron Nano V2 VL](/models/nemotron/nemotron-nano-v2-vl), [Nemotron-3 Nano Omni](/models/nemotron/nemotron-3-omni) | +| **OLMoE** | [OLMoE](/models/olmoe/olmoe) | +| **Qwen** | [Qwen](/models/qwen/qwen), [Qwen3-MoE](/models/qwen/qwen3-moe), [Qwen3-Next](/models/qwen/qwen3-next), [Qwen2.5-VL](/models/qwen/qwen2.5-vl), [Qwen3-VL](/models/qwen/qwen3-vl), [Qwen3.5 / 3.6](/models/qwen/qwen35-vl), [Qwen2-Audio](/models/qwen/qwen2-audio), [Qwen2.5-Omni](/models/qwen/qwen25-omni), [Qwen3-Omni](/models/qwen/qwen3-omni), [Qwen3-ASR](/models/qwen/qwen3-asr) | +| **Sarvam** | [Sarvam](/models/sarvam/sarvam) | +| **StepFun** | [Step-3.5-Flash](/models/stepfun/step35) | + +## Quick Navigation + +### I want to + +**Find model-specific docs** +-> Browse the family index above or use the navigation for the model's family. + +**Convert models between formats** +-> See [Bridge Guide](/get-started/quickstart/hugging-face-conversion) for Hugging Face to and from Megatron +conversion basics. Model pages include model-specific commands where available. + +**Get started with training** +-> See [Training Documentation](/training) for training guides and +[Recipe Usage](/training/recipe-usage) for pre-configured training recipes. + +**Add support for a new model** +-> Refer to [Adding New Models](/use-cases/model-development/add-new-model). + +## Model Documentation Structure + +Each model documentation page typically includes: + +1. **Model Overview** - Architecture and key features +2. **Available Variants** - Supported model sizes and configurations +3. **Conversion Examples** - Converting between Hugging Face and Megatron formats +4. **Training Recipes** - Links to training configurations and examples +5. **Architecture Details** - Model-specific features and configurations + +## Model Support Overview + +### Decoder-Only and Hybrid Backbones + +- Bailing, DeepSeek, Falcon, Gemma, GLM, GPT-OSS, Kimi, Llama, MiniMax, Mistral, Moonlight, Nemotron, OLMoE, Qwen, Sarvam, StepFun, and Xiaomi-MiMo +- MoE and hybrid variants including Bailing, DeepSeek, GLM, GPT-OSS, MiniMax, Nemotron-3, OLMoE, Qwen3-MoE, Qwen3-Next, and Sarvam + +### Multimodal Variants + +- Gemma 3 VL and Gemma 4 VL +- GLM-4.5V +- Kimi-K2.5-VL +- Ministral 3 +- Nemotron Nano V2 VL and Nemotron-3 Nano Omni +- Qwen2-Audio, Qwen2.5-VL, Qwen2.5-Omni, Qwen3-VL, Qwen3.5 / 3.6, Qwen3-Omni, and Qwen3-ASR + +## Related Documentation + +- **[Main Documentation](/resources/documentation-map)** - Return to main documentation +- **[Bridge Guide](/get-started/quickstart/hugging-face-conversion)** - Hugging Face to and from Megatron conversion +- **[Bridge Tech Details](/about/architecture)** - Technical details of the bridge system +- **[Training Documentation](/training)** - Comprehensive training guides +- **[Adding New Models](/use-cases/model-development/add-new-model)** - Extending model support +- **[Recipe Usage](/training/recipe-usage)** - Using pre-configured training recipes + +## Conversion Support + +All model pages document support for one or both conversion directions: + +- **Hugging Face -> Megatron Bridge**: Load pretrained weights for training +- **Megatron Bridge -> Hugging Face**: Export trained models for deployment + +Conversion features: + +- Automatic architecture detection +- Parallelism-aware conversion (TP/PP/VPP/CP/EP) +- Streaming and memory-efficient transfers +- Verification mechanisms for conversion accuracy + +Refer to the [Bridge Guide](/get-started/quickstart/hugging-face-conversion) for detailed conversion instructions. diff --git a/docs/models/kimi/index.md b/docs/models/kimi/index.md deleted file mode 100644 index 945d29b26e..0000000000 --- a/docs/models/kimi/index.md +++ /dev/null @@ -1,15 +0,0 @@ -# Kimi - -Kimi model documentation is organized by model variant. - -```{toctree} -:hidden: - -kimi-k2.md -kimi-k25-vl.md -``` - -| Variant | Guide | -|---------|-------| -| Kimi K2 | [kimi-k2.md](kimi-k2.md) | -| Kimi-K2.5-VL | [kimi-k25-vl.md](kimi-k25-vl.md) | diff --git a/docs/models/kimi/index.mdx b/docs/models/kimi/index.mdx new file mode 100644 index 0000000000..ccd1af6897 --- /dev/null +++ b/docs/models/kimi/index.mdx @@ -0,0 +1,13 @@ +--- +title: "Kimi" +description: "" +--- + +# Kimi + +Kimi model documentation is organized by model variant. + +| Variant | Guide | +|---------|-------| +| Kimi K2 | [kimi-k2.md](/models/kimi/kimi-k2) | +| Kimi-K2.5-VL | [kimi-k25-vl.md](/models/kimi/kimi-k25-vl) | diff --git a/docs/models/kimi/kimi-k2.md b/docs/models/kimi/kimi-k2.mdx similarity index 96% rename from docs/models/kimi/kimi-k2.md rename to docs/models/kimi/kimi-k2.mdx index 2b72e337bd..c078b269be 100644 --- a/docs/models/kimi/kimi-k2.md +++ b/docs/models/kimi/kimi-k2.mdx @@ -1,3 +1,8 @@ +--- +title: "Kimi K2" +description: "" +--- + # Kimi K2 [Kimi K2](https://huggingface.co/moonshotai/Kimi-K2-Instruct) is a large sparse MoE language model from Moonshot AI. Megatron Bridge supports Kimi K2 through the `KimiK2Bridge`, which reuses the common MLA/MoE mapping pattern used by DeepSeek-style architectures. diff --git a/docs/models/kimi/kimi-k25-vl.md b/docs/models/kimi/kimi-k25-vl.mdx similarity index 97% rename from docs/models/kimi/kimi-k25-vl.md rename to docs/models/kimi/kimi-k25-vl.mdx index 405d5df8d0..90bb15b639 100644 --- a/docs/models/kimi/kimi-k25-vl.md +++ b/docs/models/kimi/kimi-k25-vl.mdx @@ -1,3 +1,8 @@ +--- +title: "Kimi-K2.5-VL" +description: "" +--- + # Kimi-K2.5-VL [Kimi-K2.5-VL](https://huggingface.co/moonshotai/Kimi-K2.5) is a Moonshot AI vision-language model with a Kimi K2-style MoE + MLA language backbone and a vision encoder. Megatron Bridge supports it through a dedicated VLM bridge and provider. @@ -22,4 +27,3 @@ For conversion, inference, SFT, PEFT, Slurm launch scripts, and dataset notes, s - Bridge implementation: [`src/megatron/bridge/models/kimi_vl`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/src/megatron/bridge/models/kimi_vl) - Recipes: [`src/megatron/bridge/recipes/kimi_vl/kimi_k25_vl.py`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/recipes/kimi_vl/kimi_k25_vl.py) - Examples: [`examples/models/kimi/kimi_k25_vl`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/models/kimi/kimi_k25_vl) - diff --git a/docs/models/llama/index.md b/docs/models/llama/index.mdx similarity index 66% rename from docs/models/llama/index.md rename to docs/models/llama/index.mdx index 26d0150817..98ad245704 100644 --- a/docs/models/llama/index.md +++ b/docs/models/llama/index.mdx @@ -1,18 +1,16 @@ +--- +title: "Llama" +description: "" +--- + # Llama Llama model documentation is organized by model variant. -```{toctree} -:hidden: - -llama2.md -llama3.md -``` - | Variant | Guide | |---------|-------| -| Llama 2 | [llama2.md](llama2.md) | -| Llama 3 | [llama3.md](llama3.md) | +| Llama 2 | [llama2.md](/models/llama/llama2) | +| Llama 3 | [llama3.md](/models/llama/llama3) | ## Recipes diff --git a/docs/models/llama/llama2.md b/docs/models/llama/llama2.mdx similarity index 97% rename from docs/models/llama/llama2.md rename to docs/models/llama/llama2.mdx index 7910091900..cf880942c4 100644 --- a/docs/models/llama/llama2.md +++ b/docs/models/llama/llama2.mdx @@ -1,3 +1,8 @@ +--- +title: "Llama 2" +description: "" +--- + # Llama 2 [Llama 2](https://huggingface.co/meta-llama/Llama-2-7b-hf) is Meta's decoder-only transformer model family. Megatron Bridge supports Llama 2 through the shared Llama bridge for the Hugging Face `LlamaForCausalLM` architecture. @@ -41,4 +46,3 @@ uv run python examples/conversion/hf_to_megatron_generate_text.py \ ## Related Implementation - Bridge implementation: [`src/megatron/bridge/models/llama/llama_bridge.py`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/models/llama/llama_bridge.py) - diff --git a/docs/models/llama/llama3.md b/docs/models/llama/llama3.mdx similarity index 96% rename from docs/models/llama/llama3.md rename to docs/models/llama/llama3.mdx index e6adaea945..3724447718 100644 --- a/docs/models/llama/llama3.md +++ b/docs/models/llama/llama3.mdx @@ -1,3 +1,8 @@ +--- +title: "Llama 3" +description: "" +--- + # Llama 3 [Meta's Llama](https://www.llama.com/models/llama-3/) builds on the general transformer decoder framework with some key additions such as pre-normalization, SwiGLU activations, and Rotary Positional Embeddings (RoPE). More information is available in the companion paper ["Llama: Open and Efficient Foundation Language Models"](https://arxiv.org/abs/2302.13971). With a wide variety of model sizes - Llama has options for every inference budget. @@ -24,7 +29,7 @@ All models support both pretraining and finetuning with full parameter updates o ## Conversion with ๐Ÿค— Hugging Face -### Load HF โ†’ Megatron +### Load HF -> Megatron ```python from megatron.bridge import AutoBridge @@ -48,7 +53,7 @@ uv run python examples/conversion/convert_checkpoints.py import \ --megatron-path /checkpoints/llama31_8b_megatron ``` -### Export Megatron โ†’ HF +### Export Megatron -> HF ```python from megatron.bridge import AutoBridge @@ -78,7 +83,7 @@ For more details, see [examples/conversion/hf_to_megatron_generate_text.py](http ## Recipes -See: [bridge.recipes.llama.llama3](../../apidocs/bridge/bridge.recipes.llama.llama3.md) +See: [bridge.recipes.llama.llama3](/reference/api) ### Available Recipes @@ -225,6 +230,6 @@ cfg = llama3_70b_peft_config( - The Llama 3 Herd of Models: [arXiv:2407.21783](https://arxiv.org/abs/2407.21783) ## Related Docs -- Recipe usage: [Recipe usage](../../recipe-usage.md) -- Customizing the training recipe configuration: [Configuration overview](../../training/config-container-overview.md) -- Training entry points: [Entry points](../../training/entry-points.md) +- Recipe usage: [Recipe usage](/training/recipe-usage) +- Customizing the training recipe configuration: [Configuration overview](/training/config-container-overview) +- Training entry points: [Entry points](/training/entry-points) diff --git a/docs/models/mimo/index.md b/docs/models/mimo/index.mdx similarity index 56% rename from docs/models/mimo/index.md rename to docs/models/mimo/index.mdx index 1c394956cb..25761eb4cc 100644 --- a/docs/models/mimo/index.md +++ b/docs/models/mimo/index.mdx @@ -1,13 +1,12 @@ +--- +title: "Xiaomi-MiMo" +description: "" +--- + # Xiaomi-MiMo Xiaomi-MiMo model documentation is organized by model variant. -```{toctree} -:hidden: - -mimo.md -``` - | Variant | Guide | |---------|-------| -| Xiaomi-MiMo | [mimo.md](mimo.md) | +| Xiaomi-MiMo | [mimo.md](/models/mimo/mimo) | diff --git a/docs/models/mimo/mimo.md b/docs/models/mimo/mimo.mdx similarity index 96% rename from docs/models/mimo/mimo.md rename to docs/models/mimo/mimo.mdx index 978d609383..fb85612272 100644 --- a/docs/models/mimo/mimo.md +++ b/docs/models/mimo/mimo.mdx @@ -1,3 +1,8 @@ +--- +title: "Xiaomi-MiMo" +description: "" +--- + # Xiaomi-MiMo Xiaomi-MiMo models use a Qwen2-style causal language backbone with Multi-Token Prediction support. Megatron Bridge supports MiMo causal language models through `MimoBridge`, which extends the Qwen2 bridge and adds MTP weight mappings. @@ -21,4 +26,3 @@ General MiMo and heterogeneous multimodal training examples live under [`example - MiMo bridge: [`src/megatron/bridge/models/mimo/mimo_bridge.py`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/models/mimo/mimo_bridge.py) - Megatron-MiMo provider infrastructure: [`src/megatron/bridge/models/megatron_mimo`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/src/megatron/bridge/models/megatron_mimo) - diff --git a/docs/models/minimax/index.md b/docs/models/minimax/index.md deleted file mode 100644 index fec9392bfe..0000000000 --- a/docs/models/minimax/index.md +++ /dev/null @@ -1,13 +0,0 @@ -# MiniMax - -MiniMax model documentation is organized by model variant. - -```{toctree} -:hidden: - -minimax-m2.md -``` - -| Variant | Guide | -|---------|-------| -| MiniMax-M2 / M2.5 / M2.7 | [minimax-m2.md](minimax-m2.md) | diff --git a/docs/models/minimax/index.mdx b/docs/models/minimax/index.mdx new file mode 100644 index 0000000000..a1b5d6684e --- /dev/null +++ b/docs/models/minimax/index.mdx @@ -0,0 +1,12 @@ +--- +title: "MiniMax" +description: "" +--- + +# MiniMax + +MiniMax model documentation is organized by model variant. + +| Variant | Guide | +|---------|-------| +| MiniMax-M2 / M2.5 / M2.7 | [minimax-m2.md](/models/minimax/minimax-m2) | diff --git a/docs/models/minimax/minimax-m2.md b/docs/models/minimax/minimax-m2.mdx similarity index 97% rename from docs/models/minimax/minimax-m2.md rename to docs/models/minimax/minimax-m2.mdx index 3ade09d385..cc6d3baaf5 100644 --- a/docs/models/minimax/minimax-m2.md +++ b/docs/models/minimax/minimax-m2.mdx @@ -1,3 +1,8 @@ +--- +title: "MiniMax-M2" +description: "" +--- + # MiniMax-M2 [MiniMax-M2](https://huggingface.co/MiniMaxAI/MiniMax-M2) is a large sparse MoE language model from MiniMaxAI. Megatron Bridge supports MiniMax-M2 and architecture-compatible M2.5/M2.7 checkpoints through the `MiniMaxM2Bridge`. @@ -26,4 +31,3 @@ For Slurm conversion, inference, hardware requirements, and validated parallelis - Bridge implementation: [`src/megatron/bridge/models/minimax_m2`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/src/megatron/bridge/models/minimax_m2) - Examples: [`examples/models/minimax/minimax_m2`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/models/minimax/minimax_m2) - diff --git a/docs/models/mistral/index.md b/docs/models/mistral/index.md deleted file mode 100644 index dfaf9cb3f0..0000000000 --- a/docs/models/mistral/index.md +++ /dev/null @@ -1,15 +0,0 @@ -# Mistral - -Mistral model documentation is organized by model variant, including Ministral. - -```{toctree} -:hidden: - -mistral.md -ministral3.md -``` - -| Variant | Guide | -|---------|-------| -| Mistral | [mistral.md](mistral.md) | -| Ministral 3 | [ministral3.md](ministral3.md) | diff --git a/docs/models/mistral/index.mdx b/docs/models/mistral/index.mdx new file mode 100644 index 0000000000..71f7ab96d3 --- /dev/null +++ b/docs/models/mistral/index.mdx @@ -0,0 +1,13 @@ +--- +title: "Mistral" +description: "" +--- + +# Mistral + +Mistral model documentation is organized by model variant, including Ministral. + +| Variant | Guide | +|---------|-------| +| Mistral | [mistral.md](/models/mistral/mistral) | +| Ministral 3 | [ministral3.md](/models/mistral/ministral3) | diff --git a/docs/models/mistral/ministral3.md b/docs/models/mistral/ministral3.mdx similarity index 91% rename from docs/models/mistral/ministral3.md rename to docs/models/mistral/ministral3.mdx index b2c3661fce..8b567ffad7 100644 --- a/docs/models/mistral/ministral3.md +++ b/docs/models/mistral/ministral3.mdx @@ -1,3 +1,8 @@ +--- +title: "Ministral 3" +description: "" +--- + # Ministral 3 [Mistral AI's Ministral 3](https://huggingface.co/collections/mistralai/ministral-3) is part of the Mistral family and is designed for edge-optimized vision-language deployment across various hardware configurations. The Ministral 3 architecture combines a powerful language model with a vision encoder for multimodal understanding. @@ -6,9 +11,9 @@ Ministral 3 models support multimodal tasks including image captioning, visual q Ministral 3 models are supported via the Bridge system with auto-detected configuration and weight mapping. -```{important} + Please upgrade to `transformers` v5 and upgrade `mistral-common` in order to use the Ministral 3 models. -``` + ## Available Models @@ -63,7 +68,7 @@ For checkpoint conversion, inference, finetuning recipes, and step-by-step train - Ministral 3 14B Instruct: https://huggingface.co/mistralai/Ministral-3-14B-Instruct-2512 ## Related Docs -- Related Mistral variant: [Mistral](mistral.md) -- Recipe usage: [Recipe usage](../../recipe-usage.md) -- Customizing the training recipe configuration: [Configuration overview](../../training/config-container-overview.md) -- Training entry points: [Entry points](../../training/entry-points.md) +- Related Mistral variant: [Mistral](/models/mistral/mistral) +- Recipe usage: [Recipe usage](/training/recipe-usage) +- Customizing the training recipe configuration: [Configuration overview](/training/config-container-overview) +- Training entry points: [Entry points](/training/entry-points) diff --git a/docs/models/mistral/mistral.md b/docs/models/mistral/mistral.mdx similarity index 91% rename from docs/models/mistral/mistral.md rename to docs/models/mistral/mistral.mdx index 72340866e2..a0f33bf606 100644 --- a/docs/models/mistral/mistral.md +++ b/docs/models/mistral/mistral.mdx @@ -1,3 +1,8 @@ +--- +title: "Mistral" +description: "" +--- + # Mistral [Mistral AI](https://mistral.ai/) develops frontier large language models with a focus on efficiency and performance. The Mistral family includes both dense and Mixture-of-Experts architectures, featuring innovations like sliding window attention and efficient context handling. @@ -25,7 +30,7 @@ Additional Mistral models (including MoE variants like Mixtral) may be supported ## Conversion with ๐Ÿค— Hugging Face -### Load HF โ†’ Megatron +### Load HF -> Megatron ```python from megatron.bridge import AutoBridge @@ -49,7 +54,7 @@ uv run python examples/conversion/convert_checkpoints.py import \ --megatron-path /checkpoints/mistral_small_24b_megatron ``` -### Export Megatron โ†’ HF +### Export Megatron -> HF ```python from megatron.bridge import AutoBridge @@ -97,7 +102,7 @@ Training recipes for Mistral models are not currently available. The Bridge supp - Mistral Documentation: https://docs.mistral.ai/ ## Related Docs -- Related Mistral variant: [Ministral 3](ministral3.md) -- Recipe usage: [Recipe usage](../../recipe-usage.md) -- Customizing the training recipe configuration: [Configuration overview](../../training/config-container-overview.md) -- Training entry points: [Entry points](../../training/entry-points.md) +- Related Mistral variant: [Ministral 3](/models/mistral/ministral3) +- Recipe usage: [Recipe usage](/training/recipe-usage) +- Customizing the training recipe configuration: [Configuration overview](/training/config-container-overview) +- Training entry points: [Entry points](/training/entry-points) diff --git a/docs/models/moonlight/index.md b/docs/models/moonlight/index.mdx similarity index 75% rename from docs/models/moonlight/index.md rename to docs/models/moonlight/index.mdx index ee221d0595..4416d8f033 100644 --- a/docs/models/moonlight/index.md +++ b/docs/models/moonlight/index.mdx @@ -1,16 +1,15 @@ +--- +title: "Moonlight" +description: "" +--- + # Moonlight Moonlight model documentation is organized by model variant. -```{toctree} -:hidden: - -moonlight.md -``` - | Variant | Guide | |---------|-------| -| Moonlight | [moonlight.md](moonlight.md) | +| Moonlight | [moonlight.md](/models/moonlight/moonlight) | ## Recipes diff --git a/docs/models/moonlight/moonlight.md b/docs/models/moonlight/moonlight.mdx similarity index 93% rename from docs/models/moonlight/moonlight.md rename to docs/models/moonlight/moonlight.mdx index 67648e1be1..eb801587fa 100644 --- a/docs/models/moonlight/moonlight.md +++ b/docs/models/moonlight/moonlight.mdx @@ -1,3 +1,8 @@ +--- +title: "Moonlight" +description: "" +--- + # Moonlight [Moonlight](https://huggingface.co/moonshotai/Moonlight-16B-A3B) is a 16B-parameter Mixture-of-Experts (MoE) model from **Moonshot AI** trained with 5.7T tokens using the innovative **Muon optimizer**. While Moonlight shares the same architecture as DeepSeek-V3 (featuring Multi-head Latent Attention and MoE), it is a distinct model that advances the Pareto frontier of performance vs training FLOPs through the use of Muon, which is ~2ร— more sample efficient than Adam with compute optimal training. @@ -22,7 +27,7 @@ Moonlight models are supported via the Bridge system with specialized configurat Moonlight shares the same architecture as DeepSeek-V3, which enables compatibility with various inference engines like vLLM and SGLang. The model can be loaded from HuggingFace or used with Megatron checkpoints. -### Load HF โ†’ Megatron +### Load HF -> Megatron ```python from megatron.bridge import AutoBridge @@ -40,7 +45,7 @@ provider.finalize() model = provider.provide_distributed_model(wrap_with_ddp=False) ``` -### Export Megatron โ†’ HF +### Export Megatron -> HF ```python # Convert from a Megatron checkpoint directory to HF format bridge.export_ckpt( @@ -55,7 +60,7 @@ bridge.export_ckpt( ## Recipes -See: [bridge.recipes.moonlight](../../apidocs/bridge/bridge.recipes.moonlight.md) +See: [bridge.recipes.moonlight](/reference/api) ### Available Recipes @@ -157,7 +162,6 @@ cfg = moonlight_16b_peft_config( ## Related docs -- Recipe usage and customization: [Recipe usage](../../recipe-usage.md) -- Training configuration: [Configuration overview](../../training/config-container-overview.md) -- Training entry points: [Entry points](../../training/entry-points.md) - +- Recipe usage and customization: [Recipe usage](/training/recipe-usage) +- Training configuration: [Configuration overview](/training/config-container-overview) +- Training entry points: [Entry points](/training/entry-points) diff --git a/docs/models/nemotron/index.md b/docs/models/nemotron/index.md deleted file mode 100644 index 66089cf1a2..0000000000 --- a/docs/models/nemotron/index.md +++ /dev/null @@ -1,28 +0,0 @@ -# Nemotron - -Nemotron model documentation is organized by model variant. - -```{toctree} -:hidden: - -llama-nemotron.md -nemotronh.md -nemotron3-nano.md -nemotron3-super.md -nemotron-nano-v2-vl.md -nemotron-3-omni.md -``` - -| Variant | Guide | -|---------|-------| -| Llama Nemotron | [llama-nemotron.md](llama-nemotron.md) | -| Nemotron H and Nemotron Nano v2 | [nemotronh.md](nemotronh.md) | -| Nemotron-3 Nano | [nemotron3-nano.md](nemotron3-nano.md) | -| Nemotron-3 Super | [nemotron3-super.md](nemotron3-super.md) | -| Nemotron Nano V2 VL | [nemotron-nano-v2-vl.md](nemotron-nano-v2-vl.md) | -| Nemotron-3 Nano Omni | [nemotron-3-omni.md](nemotron-3-omni.md) | - -## Examples - -- Nemotron-3 examples: [`examples/models/nemotron/nemotron_3/README.md`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/models/nemotron/nemotron_3/README.md) -- Nemotron-3 Omni examples: [`examples/models/nemotron/nemotron_3_omni/README.md`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/models/nemotron/nemotron_3_omni/README.md) diff --git a/docs/models/nemotron/index.mdx b/docs/models/nemotron/index.mdx new file mode 100644 index 0000000000..1dfd243f06 --- /dev/null +++ b/docs/models/nemotron/index.mdx @@ -0,0 +1,22 @@ +--- +title: "Nemotron" +description: "" +--- + +# Nemotron + +Nemotron model documentation is organized by model variant. + +| Variant | Guide | +|---------|-------| +| Llama Nemotron | [llama-nemotron.md](/models/nemotron/llama-nemotron) | +| Nemotron H and Nemotron Nano v2 | [nemotronh.md](/models/nemotron/nemotronh) | +| Nemotron-3 Nano | [nemotron3-nano.md](/models/nemotron/nemotron3-nano) | +| Nemotron-3 Super | [nemotron3-super.md](/models/nemotron/nemotron3-super) | +| Nemotron Nano V2 VL | [nemotron-nano-v2-vl.md](/models/nemotron/nemotron-nano-v2-vl) | +| Nemotron-3 Nano Omni | [nemotron-3-omni.md](/models/nemotron/nemotron-3-omni) | + +## Examples + +- Nemotron-3 examples: [`examples/models/nemotron/nemotron_3/README.md`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/models/nemotron/nemotron_3/README.md) +- Nemotron-3 Omni examples: [`examples/models/nemotron/nemotron_3_omni/README.md`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/examples/models/nemotron/nemotron_3_omni/README.md) diff --git a/docs/models/nemotron/llama-nemotron.md b/docs/models/nemotron/llama-nemotron.mdx similarity index 94% rename from docs/models/nemotron/llama-nemotron.md rename to docs/models/nemotron/llama-nemotron.mdx index ce1cecad23..d542395178 100644 --- a/docs/models/nemotron/llama-nemotron.md +++ b/docs/models/nemotron/llama-nemotron.mdx @@ -1,3 +1,8 @@ +--- +title: "Llama Nemotron" +description: "" +--- + # Llama Nemotron [Llama Nemotron](https://huggingface.co/collections/nvidia/llama-nemotron) is NVIDIA's family of large language models derived from Meta's Llama architecture, post-trained for enhanced reasoning, human chat preferences, and agentic tasks such as RAG and tool calling. The models feature neural architecture search (NAS) optimizations for improved efficiency and accuracy trade-offs. @@ -32,7 +37,7 @@ All models are ready for commercial use and support context lengths up to 128K t ## Conversion with ๐Ÿค— Hugging Face -### Load HF โ†’ Megatron +### Load HF -> Megatron ```python from megatron.bridge import AutoBridge @@ -62,7 +67,7 @@ uv run python examples/conversion/convert_checkpoints.py import \ --trust-remote-code ``` -### Export Megatron โ†’ HF +### Export Megatron -> HF ```python from megatron.bridge import AutoBridge @@ -117,7 +122,6 @@ Training recipes for Llama Nemotron models are not currently available. - Llama Nemotron Post-Training Dataset: https://huggingface.co/nvidia/Llama-Nemotron-Post-Training-Dataset ## Related Docs -- Recipe usage: [Recipe usage](../../recipe-usage.md) -- Customizing the training recipe configuration: [Configuration overview](../../training/config-container-overview.md) -- Training entry points: [Entry points](../../training/entry-points.md) - +- Recipe usage: [Recipe usage](/training/recipe-usage) +- Customizing the training recipe configuration: [Configuration overview](/training/config-container-overview) +- Training entry points: [Entry points](/training/entry-points) diff --git a/docs/models/nemotron/nemotron-3-omni.md b/docs/models/nemotron/nemotron-3-omni.mdx similarity index 87% rename from docs/models/nemotron/nemotron-3-omni.md rename to docs/models/nemotron/nemotron-3-omni.mdx index d6207b714d..13b48d2066 100644 --- a/docs/models/nemotron/nemotron-3-omni.md +++ b/docs/models/nemotron/nemotron-3-omni.mdx @@ -1,3 +1,8 @@ +--- +title: "Nemotron-3 Nano Omni" +description: "" +--- + # Nemotron-3 Nano Omni [NVIDIA Nemotron-3 Nano Omni](https://huggingface.co/nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16) @@ -7,19 +12,19 @@ language backbone with a RADIO vision tower (static-resolution image path or dynamic-resolution temporal video embedder) and a Parakeet sound encoder. -NeMo Megatron Bridge supports HFโ†”Megatron conversion, full SFT, and LoRA +NeMo Megatron Bridge supports HF to and from Megatron conversion, full SFT, and LoRA PEFT on image-text and audio-video-text datasets. The finetuned model can be re-exported to ๐Ÿค— Hugging Face format for downstream evaluation or deployment. -```{important} + **Day-0 release.** This model is supported on dedicated public branches: | Repo | Branch | |---|---| | Megatron-Bridge | [`nemotron_3_omni`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/nemotron_3_omni) | | Megatron-LM (submodule at `3rdparty/Megatron-LM`) | [`nemotron_3_omni`](https://github.com/NVIDIA/Megatron-LM/tree/nemotron_3_omni) | -``` + For the full setup, conversion, inference, training, evaluation, and LoRA merge / adapter export workflows, see diff --git a/docs/models/nemotron/nemotron-nano-v2-vl.md b/docs/models/nemotron/nemotron-nano-v2-vl.mdx similarity index 98% rename from docs/models/nemotron/nemotron-nano-v2-vl.md rename to docs/models/nemotron/nemotron-nano-v2-vl.mdx index 0de859b0ad..bbc6a157d1 100644 --- a/docs/models/nemotron/nemotron-nano-v2-vl.md +++ b/docs/models/nemotron/nemotron-nano-v2-vl.mdx @@ -1,3 +1,8 @@ +--- +title: "Nemotron Nano V2 VL" +description: "" +--- + # Nemotron Nano V2 VL NVIDIA Nemotron Nano v2 VL is an open 12B multimodal reasoning model for document intelligence and video understanding. @@ -10,13 +15,13 @@ NeMo Megatron Bridge supports finetuning this model (including LoRA finetuning) datasets. The finetuned model can be converted back to the ๐Ÿค— Hugging Face format for downstream evaluation. -```{important} + Please use the custom container `nvcr.io/nvidia/nemo:25.09.nemotron_nano_v2_vl` when working with this model. Run all commands from `/opt/Megatron-Bridge` (e.g. `docker run -w /opt/Megatron-Bridge ...`) -``` + -```{tip} + We use the following environment variables throughout this page - `HF_MODEL_PATH=nvidia/NVIDIA-Nemotron-Nano-12B-v2-VL-BF16` - `MEGATRON_MODEL_PATH=/models/NVIDIA-Nemotron-Nano-12B-v2-VL-BF16` (feel free to set your own path) @@ -24,11 +29,11 @@ We use the following environment variables throughout this page Unless explicitly stated, any megatron model path in the commands below should NOT contain the iteration number `iter_xxxxxx`. For more details on checkpointing, please see [here](https://docs.nvidia.com/nemo/megatron-bridge/latest/training/checkpointing.html#checkpoint-contents) -``` + ## Conversion with ๐Ÿค— Hugging Face -### Import HF โ†’ Megatron +### Import HF -> Megatron To import the HF model to your desired `$MEGATRON_MODEL_PATH`, run the following command. ```bash uv run python examples/conversion/convert_checkpoints.py import \ @@ -37,7 +42,7 @@ uv run python examples/conversion/convert_checkpoints.py import \ --trust-remote-code ``` -### Export Megatron โ†’ HF +### Export Megatron -> HF You can export a trained model with the following command. ```bash uv run python examples/conversion/convert_checkpoints.py export \ diff --git a/docs/models/nemotron/nemotron3-nano.md b/docs/models/nemotron/nemotron3-nano.mdx similarity index 95% rename from docs/models/nemotron/nemotron3-nano.md rename to docs/models/nemotron/nemotron3-nano.mdx index 981a37405c..c7794b8dcb 100644 --- a/docs/models/nemotron/nemotron3-nano.md +++ b/docs/models/nemotron/nemotron3-nano.mdx @@ -1,22 +1,27 @@ +--- +title: "Nemotron 3 Nano" +description: "" +--- + # Nemotron 3 Nano [Nemotron 3 Nano](https://huggingface.co/collections/nvidia/nvidia-nemotron-v3) is a large language model (LLM) trained from scratch by NVIDIA, and designed as a unified model for both reasoning and non-reasoning tasks. The model employs a hybrid Mixture-of-Experts (MoE) architecture, consisting of 23 Mamba-2 and MoE layers, along with 6 Attention layers. Each MoE layer includes 128 experts plus 1 shared expert, with 5 experts activated per token. The model has 3.5B active parameters and 30B parameters in total. NeMo Megatron Bridge supports pretraining, full parameters finetuning, and LoRA finetuning this model. The finetuned model can be converted back to the ๐Ÿค— Hugging Face format for downstream evaluation. -```{important} + Run all commands from `/opt/Megatron-Bridge` (e.g. `docker run -w /opt/Megatron-Bridge ...`) -``` + -```{tip} + We use the following environment variables throughout this page - `HF_MODEL_ID=nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16` - `MEGATRON_MODEL_PATH=/models/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16` (feel free to set your own path) -``` + ## Conversion with ๐Ÿค— Hugging Face -### Import HF โ†’ Megatron +### Import HF -> Megatron To import the HF model to your desired `$MEGATRON_MODEL_PATH`, run the following command. ```bash uv run python examples/conversion/convert_checkpoints.py import \ @@ -25,7 +30,7 @@ uv run python examples/conversion/convert_checkpoints.py import \ --trust-remote-code ``` -### Export Megatron โ†’ HF +### Export Megatron -> HF ```bash uv run python examples/conversion/convert_checkpoints.py export \ --hf-model $HF_MODEL_ID \ @@ -66,7 +71,7 @@ checkpoint.pretrained_checkpoint=/path/to/output/megatron/ckpt Notes: - Default parallelism TP=1, EP=8, PP=1, CP=1. Running this recipe requires at least 2 H100 nodes (16 GPUs). - By default, the [SQuAD](https://huggingface.co/datasets/rajpurkar/squad) dataset is used. To use customerized dataset, see this [tutorial](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/tutorials/recipes/llama#quickstart) -- Fine-tuning requires a pretrained megatron checkpoint, which can be obtained in "Import HF โ†’ Megatron" section above +- Fine-tuning requires a pretrained megatron checkpoint, which can be obtained in "Import HF -> Megatron" section above ### LoRA Fine-Tuning diff --git a/docs/models/nemotron/nemotron3-super.md b/docs/models/nemotron/nemotron3-super.mdx similarity index 97% rename from docs/models/nemotron/nemotron3-super.md rename to docs/models/nemotron/nemotron3-super.mdx index 2e227525fb..942983021a 100644 --- a/docs/models/nemotron/nemotron3-super.md +++ b/docs/models/nemotron/nemotron3-super.mdx @@ -1,11 +1,16 @@ +--- +title: "Nemotron 3 Super" +description: "" +--- + # Nemotron 3 Super [Nemotron 3 Super](https://huggingface.co/collections/nvidia/nvidia-nemotron-v3)is a large language model (LLM) trained by NVIDIA, designed to deliver strong agentic, reasoning, and conversational capabilities. It is employs a hybrid **Latent Mixture-of-Experts (LatentMoE)** architecture, utilizing interleaved Mamba-2 and MoE layers, along with select Attention layers. Distinct from the Nano model, the Super model incorporates **Multi-Token Prediction (MTP)** layers for faster text generation and improved quality, and it is trained using **NVFP4** quantization to maximize compute efficiency. The model has **12B active parameters** and **120B parameters in total**. NeMo Megatron Bridge supports pretraining, full parameters finetuning, and LoRA finetuning this model. The finetuned model can be converted back to the ๐Ÿค— Hugging Face format for downstream evaluation. -```{important} + Run all commands from `/opt/Megatron-Bridge` (e.g. `docker run -w /opt/Megatron-Bridge ...`) -``` + ## Getting the Latest Code @@ -47,7 +52,7 @@ docker run --rm -it \ ## Conversion with ๐Ÿค— Hugging Face -### Import HF โ†’ Megatron +### Import HF -> Megatron To import the HF model to your desired `$MEGATRON_MODEL_PATH`, use the distributed conversion script because this model uses expert parallelism. The single-process `examples/conversion/convert_checkpoints.py` script is limited to single-GPU conversion @@ -68,7 +73,7 @@ Notes: - The default parallelism is TP=1, EP=8 (Expert Parallel) - Adjust `--nproc-per-node` based on your available GPUs -### Export Megatron โ†’ HF +### Export Megatron -> HF ```bash HF_MODEL=/path/to/hf/model MEGATRON_PATH=/path/to/trained/megatron/ckpt @@ -197,7 +202,7 @@ checkpoint.pretrained_checkpoint=$MEGATRON_PATH Notes: - Default parallelism TP=4, EP=8, PP=1, CP=1 with sequence parallel enabled - By default, the [SQuAD](https://huggingface.co/datasets/rajpurkar/squad) dataset is used. -- Fine-tuning requires a pretrained Megatron checkpoint, which can be obtained from the "Import HF โ†’ Megatron" section above +- Fine-tuning requires a pretrained Megatron checkpoint, which can be obtained from the "Import HF -> Megatron" section above - Adjust `global_batch_size` and parallelism settings based on your GPU memory and requirements @@ -233,9 +238,9 @@ Notes: ## Quantization (PTQ and QAT) -```{important} + Quantization support requires the latest code from the `super-v3` branch. See [Getting the Latest Code](#getting-the-latest-code) for instructions. -``` + Nemotron 3 Super supports four quantization configurations: @@ -277,7 +282,7 @@ uv run python -m torch.distributed.run --nproc_per_node=8 examples/quantization/ Notes: - For multi-node setups (e.g. 2 nodes with 8ร— H100), increase `--pp` accordingly (e.g. `--pp 2`) and use a job scheduler like SLURM to launch across nodes. -### Export Quantized Megatron Checkpoint โ†’ HF +### Export Quantized Megatron Checkpoint -> HF After quantization, export the Megatron checkpoint back to Hugging Face format: diff --git a/docs/models/nemotron/nemotronh.md b/docs/models/nemotron/nemotronh.mdx similarity index 96% rename from docs/models/nemotron/nemotronh.md rename to docs/models/nemotron/nemotronh.mdx index 6badf9633a..8fa3fa68de 100644 --- a/docs/models/nemotron/nemotronh.md +++ b/docs/models/nemotron/nemotronh.mdx @@ -1,3 +1,8 @@ +--- +title: "Nemotron H and Nemotron Nano v2" +description: "" +--- + # Nemotron H and Nemotron Nano v2 [Nemotron H](https://huggingface.co/collections/nvidia/nemotron-h) and [Nemotron Nano v2](https://huggingface.co/collections/nvidia/nvidia-nemotron-v2) are families of **hybrid SSM-Attention models** from **NVIDIA** that combine Mamba (State Space Model) layers with traditional attention layers. These models achieve strong performance while maintaining computational efficiency through their hybrid architecture. @@ -123,7 +128,7 @@ All models are supported via the Bridge system with specialized configurations f ## Conversion with ๐Ÿค— Hugging Face -### Load HF โ†’ Megatron +### Load HF -> Megatron #### Nemotron H Models ```python @@ -172,7 +177,7 @@ model = provider.provide_distributed_model(wrap_with_ddp=False) # bridge = AutoBridge.from_hf_pretrained("nvidia/NVIDIA-Nemotron-Nano-12B-v2-Base", trust_remote_code=True) ``` -### Export Megatron โ†’ HF +### Export Megatron -> HF ```python # Convert from a Megatron checkpoint directory to HF format bridge.export_ckpt( @@ -378,10 +383,10 @@ cfg = nemotron_nano_12b_v2_peft_config( ## API Reference ### Nemotron H -- Nemotron H recipes: [bridge.recipes.nemotronh](../../apidocs/bridge/bridge.recipes.nemotronh.md) +- Nemotron H recipes: [bridge.recipes.nemotronh](/reference/api) ### Nemotron Nano v2 -- Nemotron Nano v2 recipes: [bridge.recipes.nemotronh.nemotron_nano_v2](../../apidocs/bridge/bridge.recipes.nemotronh.md) +- Nemotron Nano v2 recipes: [bridge.recipes.nemotronh.nemotron_nano_v2](/reference/api) ## Performance Optimizations @@ -435,8 +440,7 @@ Nemotron H models support several PP configurations with pre-defined layouts: ## Related Documentation -- Recipe usage and customization: [Recipe usage](../../recipe-usage.md) -- Training configuration: [Configuration overview](../../training/config-container-overview.md) -- Training entry points: [Entry points](../../training/entry-points.md) -- PEFT methods (LoRA, DoRA): [PEFT Guide](../../training/peft.md) - +- Recipe usage and customization: [Recipe usage](/training/recipe-usage) +- Training configuration: [Configuration overview](/training/config-container-overview) +- Training entry points: [Entry points](/training/entry-points) +- PEFT methods (LoRA, DoRA): [PEFT Guide](/training/peft) diff --git a/docs/models/olmoe/index.md b/docs/models/olmoe/index.mdx similarity index 77% rename from docs/models/olmoe/index.md rename to docs/models/olmoe/index.mdx index 37dda045ec..09a889cdfc 100644 --- a/docs/models/olmoe/index.md +++ b/docs/models/olmoe/index.mdx @@ -1,16 +1,15 @@ +--- +title: "OLMoE" +description: "" +--- + # OLMoE OLMoE model documentation is organized by model variant. -```{toctree} -:hidden: - -olmoe.md -``` - | Variant | Guide | |---------|-------| -| OLMoE | [olmoe.md](olmoe.md) | +| OLMoE | [olmoe.md](/models/olmoe/olmoe) | ## Recipes diff --git a/docs/models/olmoe/olmoe.md b/docs/models/olmoe/olmoe.mdx similarity index 94% rename from docs/models/olmoe/olmoe.md rename to docs/models/olmoe/olmoe.mdx index bba81c8bff..0214df7d27 100644 --- a/docs/models/olmoe/olmoe.md +++ b/docs/models/olmoe/olmoe.mdx @@ -1,3 +1,8 @@ +--- +title: "OLMoE" +description: "" +--- + # OLMoE [OLMoE](https://huggingface.co/allenai/OLMoE-1B-7B-0125) is a 7B-parameter Mixture-of-Experts (MoE) model from **Allen Institute for AI (AI2)** featuring 64 experts with top-8 routing. The model is designed to be fully open-source, with training data, code, and model weights publicly available. It's named "OLMoE-1B-7B" where 1B refers to the activated parameters and 7B refers to the total parameters. @@ -32,7 +37,7 @@ OLMoE models are supported via the Bridge system with specialized configurations ## Conversion with ๐Ÿค— Hugging Face -### Load HF โ†’ Megatron +### Load HF -> Megatron ```python from megatron.bridge import AutoBridge @@ -52,7 +57,7 @@ model = provider.provide_distributed_model(wrap_with_ddp=False) # bridge = AutoBridge.from_hf_pretrained("allenai/OLMoE-1B-7B-0924") ``` -### Export Megatron โ†’ HF +### Export Megatron -> HF ```python # Convert from a Megatron checkpoint directory to HF format bridge.export_ckpt( @@ -67,7 +72,7 @@ bridge.export_ckpt( ## Recipes -See: [bridge.recipes.olmoe](../../apidocs/bridge/bridge.recipes.olmoe.md) +See: [bridge.recipes.olmoe](/reference/api) ### Available Recipes @@ -176,7 +181,6 @@ cfg = olmoe_7b_peft_config( ## Related docs -- Recipe usage and customization: [Recipe usage](../../recipe-usage.md) -- Training configuration: [Configuration overview](../../training/config-container-overview.md) -- Training entry points: [Entry points](../../training/entry-points.md) - +- Recipe usage and customization: [Recipe usage](/training/recipe-usage) +- Training configuration: [Configuration overview](/training/config-container-overview) +- Training entry points: [Entry points](/training/entry-points) diff --git a/docs/models/qwen/index.md b/docs/models/qwen/index.md deleted file mode 100644 index ba7074c125..0000000000 --- a/docs/models/qwen/index.md +++ /dev/null @@ -1,31 +0,0 @@ -# Qwen - -Qwen model documentation is organized by model variant. - -```{toctree} -:hidden: - -qwen.md -qwen3-moe.md -qwen3-next.md -qwen2.5-vl.md -qwen3-vl.md -qwen35-vl.md -qwen2-audio.md -qwen25-omni.md -qwen3-omni.md -qwen3-asr.md -``` - -| Variant | Guide | -|---------|-------| -| Qwen | [qwen.md](qwen.md) | -| Qwen3-MoE | [qwen3-moe.md](qwen3-moe.md) | -| Qwen3-Next | [qwen3-next.md](qwen3-next.md) | -| Qwen2.5-VL | [qwen2.5-vl.md](qwen2.5-vl.md) | -| Qwen3-VL | [qwen3-vl.md](qwen3-vl.md) | -| Qwen3.5 / 3.6 | [qwen35-vl.md](qwen35-vl.md) | -| Qwen2-Audio | [qwen2-audio.md](qwen2-audio.md) | -| Qwen2.5-Omni | [qwen25-omni.md](qwen25-omni.md) | -| Qwen3-Omni | [qwen3-omni.md](qwen3-omni.md) | -| Qwen3-ASR | [qwen3-asr.md](qwen3-asr.md) | diff --git a/docs/models/qwen/index.mdx b/docs/models/qwen/index.mdx new file mode 100644 index 0000000000..cc06a39585 --- /dev/null +++ b/docs/models/qwen/index.mdx @@ -0,0 +1,21 @@ +--- +title: "Qwen" +description: "" +--- + +# Qwen + +Qwen model documentation is organized by model variant. + +| Variant | Guide | +|---------|-------| +| Qwen | [qwen.md](/models/qwen/qwen) | +| Qwen3-MoE | [qwen3-moe.md](/models/qwen/qwen3-moe) | +| Qwen3-Next | [qwen3-next.md](/models/qwen/qwen3-next) | +| Qwen2.5-VL | [qwen2.5-vl.md](/models/qwen/qwen2.5-vl) | +| Qwen3-VL | [qwen3-vl.md](/models/qwen/qwen3-vl) | +| Qwen3.5 / 3.6 | [qwen35-vl.md](/models/qwen/qwen35-vl) | +| Qwen2-Audio | [qwen2-audio.md](/models/qwen/qwen2-audio) | +| Qwen2.5-Omni | [qwen25-omni.md](/models/qwen/qwen25-omni) | +| Qwen3-Omni | [qwen3-omni.md](/models/qwen/qwen3-omni) | +| Qwen3-ASR | [qwen3-asr.md](/models/qwen/qwen3-asr) | diff --git a/docs/models/qwen/qwen.md b/docs/models/qwen/qwen.mdx similarity index 97% rename from docs/models/qwen/qwen.md rename to docs/models/qwen/qwen.mdx index d73b636d8a..20d0cb8ee1 100644 --- a/docs/models/qwen/qwen.md +++ b/docs/models/qwen/qwen.mdx @@ -1,3 +1,8 @@ +--- +title: "Qwen" +description: "" +--- + # Qwen [Qwen](https://huggingface.co/Qwen) is a family of large language models developed by Alibaba Cloud, including dense models (Qwen2, Qwen2.5, Qwen3) and Mixture-of-Experts models (Qwen3 MoE, Qwen3-Next). The models feature innovations like QK layernorm, Gated-Delta Networks, and Zero-Centered RMSNorm for improved training stability and performance. @@ -43,7 +48,7 @@ Megatron Bridge supports the following Qwen model variants: ### Conversion with ๐Ÿค— Hugging Face -#### Load HF โ†’ Megatron +#### Load HF -> Megatron ```python from megatron.bridge import AutoBridge @@ -68,7 +73,7 @@ uv run python examples/conversion/convert_checkpoints.py import \ --megatron-path /checkpoints/qwen3_next_80b_megatron ``` -#### Export Megatron โ†’ HF +#### Export Megatron -> HF ```python from megatron.bridge import AutoBridge @@ -153,7 +158,7 @@ config = qwen3_next_80b_a3b_sft_config( ### Conversion with ๐Ÿค— Hugging Face -#### Load HF โ†’ Megatron +#### Load HF -> Megatron ```python from megatron.bridge import AutoBridge @@ -178,7 +183,7 @@ uv run python examples/conversion/convert_checkpoints.py import \ --megatron-path /checkpoints/qwen3_30b_a3b_megatron ``` -#### Export Megatron โ†’ HF +#### Export Megatron -> HF ```python from megatron.bridge import AutoBridge @@ -303,7 +308,7 @@ config = qwen3_30b_a3b_peft_config( ### Conversion with ๐Ÿค— Hugging Face -#### Load HF โ†’ Megatron +#### Load HF -> Megatron ```python from megatron.bridge import AutoBridge @@ -327,7 +332,7 @@ uv run python examples/conversion/convert_checkpoints.py import \ --megatron-path /checkpoints/qwen3_8b_megatron ``` -#### Export Megatron โ†’ HF +#### Export Megatron -> HF ```python from megatron.bridge import AutoBridge @@ -433,7 +438,7 @@ config = qwen3_8b_peft_config( ### Conversion with ๐Ÿค— Hugging Face -#### Load HF โ†’ Megatron +#### Load HF -> Megatron ```python from megatron.bridge import AutoBridge @@ -457,7 +462,7 @@ uv run python examples/conversion/convert_checkpoints.py import \ --megatron-path /checkpoints/qwen25_7b_megatron ``` -#### Export Megatron โ†’ HF +#### Export Megatron -> HF ```python from megatron.bridge import AutoBridge @@ -567,6 +572,6 @@ config = qwen25_7b_peft_config( --- ## Related Docs -- Recipe usage: [Recipe usage](../../recipe-usage.md) -- Customizing the training recipe configuration: [Configuration overview](../../training/config-container-overview.md) -- Training entry points: [Entry points](../../training/entry-points.md) +- Recipe usage: [Recipe usage](/training/recipe-usage) +- Customizing the training recipe configuration: [Configuration overview](/training/config-container-overview) +- Training entry points: [Entry points](/training/entry-points) diff --git a/docs/models/qwen/qwen2-audio.md b/docs/models/qwen/qwen2-audio.mdx similarity index 96% rename from docs/models/qwen/qwen2-audio.md rename to docs/models/qwen/qwen2-audio.mdx index 7097c6a217..b82a2852f3 100644 --- a/docs/models/qwen/qwen2-audio.md +++ b/docs/models/qwen/qwen2-audio.mdx @@ -1,3 +1,8 @@ +--- +title: "Qwen2-Audio" +description: "" +--- + # Qwen2-Audio [Qwen2-Audio](https://huggingface.co/Qwen/Qwen2-Audio-7B-Instruct) is an audio-language model from the Qwen family. Megatron Bridge supports Qwen2-Audio through a dedicated audio model bridge and provider. @@ -21,4 +26,3 @@ For runnable audio inference commands and expected output, see the [Qwen2-Audio - Bridge implementation: [`src/megatron/bridge/models/qwen_audio`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/src/megatron/bridge/models/qwen_audio) - Recipes: [`src/megatron/bridge/recipes/qwen2_audio`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/src/megatron/bridge/recipes/qwen2_audio) - Examples: [`examples/models/qwen/qwen2_audio`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/models/qwen/qwen2_audio) - diff --git a/docs/models/qwen/qwen2.5-vl.md b/docs/models/qwen/qwen2.5-vl.mdx similarity index 98% rename from docs/models/qwen/qwen2.5-vl.md rename to docs/models/qwen/qwen2.5-vl.mdx index 67a0fa9009..2d4fa2f082 100644 --- a/docs/models/qwen/qwen2.5-vl.md +++ b/docs/models/qwen/qwen2.5-vl.mdx @@ -1,3 +1,8 @@ +--- +title: "Qwen2.5-VL" +description: "" +--- + # Qwen2.5-VL Qwen2.5-VL is a series of vision-language models developed by Alibaba Cloud that enable multimodal understanding across text, images, and videos. The models support various vision-language tasks including image understanding, visual question answering, and multimodal reasoning. @@ -5,7 +10,7 @@ Qwen2.5-VL is a series of vision-language models developed by Alibaba Cloud that NeMo Megatron Bridge supports finetuning Qwen2.5-VL models (3B, 7B, 32B, and 72B variants) on single-image and multi-image datasets. The finetuned model can be converted back to the ๐Ÿค— Hugging Face format for downstream evaluation. -```{tip} + We use the following environment variables throughout this page - `HF_MODEL_PATH=Qwen/Qwen2.5-VL-3B-Instruct` (it can also be set to `Qwen/Qwen2.5-VL-7B-Instruct`, `Qwen/Qwen2.5-VL-32B-Instruct`, `Qwen/Qwen2.5-VL-72B-Instruct`) - `MEGATRON_MODEL_PATH=/models/Qwen2.5-VL-3B-Instruct` (feel free to set your own path) @@ -13,11 +18,11 @@ We use the following environment variables throughout this page Unless explicitly stated, any megatron model path in the commands below should NOT contain the iteration number `iter_xxxxxx`. For more details on checkpointing, please see [here](https://docs.nvidia.com/nemo/megatron-bridge/latest/training/checkpointing.html#checkpoint-contents) -``` + ## Conversion with ๐Ÿค— Hugging Face -### Import HF โ†’ Megatron +### Import HF -> Megatron To import the HF model to your desired `$MEGATRON_MODEL_PATH`, run the following command. ```bash uv run python examples/conversion/convert_checkpoints.py import \ @@ -25,7 +30,7 @@ uv run python examples/conversion/convert_checkpoints.py import \ --megatron-path $MEGATRON_MODEL_PATH ``` -### Export Megatron โ†’ HF +### Export Megatron -> HF You can export a trained model with the following command. ```bash uv run python examples/conversion/convert_checkpoints.py export \ @@ -140,4 +145,3 @@ To change the dataset, specify `dataset.maker_name=make_raven_dataset` - Qwen2.5-VL-7B: `https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct` - Qwen2.5-VL-32B: `https://huggingface.co/Qwen/Qwen2.5-VL-32B-Instruct` - Qwen2.5-VL-72B: `https://huggingface.co/Qwen/Qwen2.5-VL-72B-Instruct` - diff --git a/docs/models/qwen/qwen25-omni.md b/docs/models/qwen/qwen25-omni.mdx similarity index 96% rename from docs/models/qwen/qwen25-omni.md rename to docs/models/qwen/qwen25-omni.mdx index 6b8dd44774..1a97232e40 100644 --- a/docs/models/qwen/qwen25-omni.md +++ b/docs/models/qwen/qwen25-omni.mdx @@ -1,3 +1,8 @@ +--- +title: "Qwen2.5-Omni" +description: "" +--- + # Qwen2.5-Omni [Qwen2.5-Omni](https://huggingface.co/Qwen/Qwen2.5-Omni-7B) is a multimodal Qwen model for image, video, audio, and text understanding. Megatron Bridge supports it through the Qwen Omni bridge. @@ -20,4 +25,3 @@ For checkpoint import/export, round-trip validation, multimodal inference, and d - Bridge implementation: [`src/megatron/bridge/models/qwen_omni/qwen25_omni_bridge.py`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/models/qwen_omni/qwen25_omni_bridge.py) - Examples: [`examples/models/qwen/qwen25_omni`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/models/qwen/qwen25_omni) - diff --git a/docs/models/qwen/qwen3-asr.md b/docs/models/qwen/qwen3-asr.mdx similarity index 96% rename from docs/models/qwen/qwen3-asr.md rename to docs/models/qwen/qwen3-asr.mdx index c1f8b05830..4ca7dc5f13 100644 --- a/docs/models/qwen/qwen3-asr.md +++ b/docs/models/qwen/qwen3-asr.mdx @@ -1,3 +1,8 @@ +--- +title: "Qwen3-ASR" +description: "" +--- + # Qwen3-ASR [Qwen3-ASR](https://huggingface.co/Qwen/Qwen3-ASR-1.7B) is an audio speech recognition model from the Qwen3 family. Megatron Bridge supports checkpoint conversion and audio inference through the Qwen3-ASR bridge. @@ -20,4 +25,3 @@ For checkpoint import/export, round-trip validation, and audio inference command - Bridge implementation: [`src/megatron/bridge/models/qwen3_asr`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/src/megatron/bridge/models/qwen3_asr) - Examples: [`examples/models/qwen/qwen3_asr`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/models/qwen/qwen3_asr) - diff --git a/docs/models/qwen/qwen3-moe.md b/docs/models/qwen/qwen3-moe.mdx similarity index 93% rename from docs/models/qwen/qwen3-moe.md rename to docs/models/qwen/qwen3-moe.mdx index 1491222a59..d65c976995 100644 --- a/docs/models/qwen/qwen3-moe.md +++ b/docs/models/qwen/qwen3-moe.mdx @@ -1,3 +1,8 @@ +--- +title: "Qwen3-MoE" +description: "" +--- + # Qwen3-MoE [Qwen3-MoE](https://huggingface.co/Qwen/Qwen3-30B-A3B) is the sparse Mixture-of-Experts branch of the Qwen3 model family. Megatron Bridge supports Qwen3-MoE through the `Qwen3MoeBridge` and dedicated pretraining, SFT, and PEFT recipes. @@ -27,5 +32,4 @@ See [`src/megatron/bridge/recipes/qwen/qwen3_moe.py`](https://github.com/NVIDIA- ## Related Implementation - Bridge implementation: [`src/megatron/bridge/models/qwen/qwen3_moe_bridge.py`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/models/qwen/qwen3_moe_bridge.py) -- Family overview: [qwen.md](qwen.md) - +- Family overview: [qwen.md](/models/qwen/qwen) diff --git a/docs/models/qwen/qwen3-next.md b/docs/models/qwen/qwen3-next.mdx similarity index 94% rename from docs/models/qwen/qwen3-next.md rename to docs/models/qwen/qwen3-next.mdx index 65ca6dd1c8..96ea9a7ed5 100644 --- a/docs/models/qwen/qwen3-next.md +++ b/docs/models/qwen/qwen3-next.mdx @@ -1,3 +1,8 @@ +--- +title: "Qwen3-Next" +description: "" +--- + # Qwen3-Next [Qwen3-Next](https://huggingface.co/Qwen/Qwen3-Next-80B-A3B-Instruct) is a Qwen3 MoE variant with Gated-Delta Networks and Multi-Token Prediction support. Megatron Bridge supports it through the `Qwen3NextBridge`. @@ -29,5 +34,4 @@ See [`src/megatron/bridge/recipes/qwen/qwen3_next.py`](https://github.com/NVIDIA ## Related Implementation - Bridge implementation: [`src/megatron/bridge/models/qwen/qwen3_next_bridge.py`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/models/qwen/qwen3_next_bridge.py) -- Family overview: [qwen.md](qwen.md) - +- Family overview: [qwen.md](/models/qwen/qwen) diff --git a/docs/models/qwen/qwen3-omni.md b/docs/models/qwen/qwen3-omni.mdx similarity index 85% rename from docs/models/qwen/qwen3-omni.md rename to docs/models/qwen/qwen3-omni.mdx index 7dc60148e7..cc7d7f62ee 100644 --- a/docs/models/qwen/qwen3-omni.md +++ b/docs/models/qwen/qwen3-omni.mdx @@ -1,3 +1,8 @@ +--- +title: "Qwen3-Omni" +description: "" +--- + # Qwen3-Omni Qwen3-Omni is a multimodal Qwen family model with text, image, video, and audio inputs. Megatron Bridge support for Qwen3-Omni reuses the existing Qwen3-VL language and vision path, and adds Qwen3-Omni-specific audio handling and checkpoint mappings. @@ -36,8 +41,8 @@ Qwen3-Omni examples are maintained here: ## Related Docs -- Related VLM: [Qwen3-VL](qwen3-vl.md) -- Related VLM: [Qwen 3.5](qwen35-vl.md) -- Recipe usage: [Recipe usage](../../recipe-usage.md) -- Customizing the training recipe configuration: [Configuration overview](../../training/config-container-overview.md) -- Training entry points: [Entry points](../../training/entry-points.md) +- Related VLM: [Qwen3-VL](/models/qwen/qwen3-vl) +- Related VLM: [Qwen 3.5](/models/qwen/qwen35-vl) +- Recipe usage: [Recipe usage](/training/recipe-usage) +- Customizing the training recipe configuration: [Configuration overview](/training/config-container-overview) +- Training entry points: [Entry points](/training/entry-points) diff --git a/docs/models/qwen/qwen3-vl.md b/docs/models/qwen/qwen3-vl.mdx similarity index 94% rename from docs/models/qwen/qwen3-vl.md rename to docs/models/qwen/qwen3-vl.mdx index 14c4a73be1..8252621901 100644 --- a/docs/models/qwen/qwen3-vl.md +++ b/docs/models/qwen/qwen3-vl.mdx @@ -1,17 +1,22 @@ +--- +title: "Qwen3-VL" +description: "" +--- + # Qwen3-VL Qwen3-VL is the latest generation of vision-language models from Alibaba Cloud, supporting multimodal understanding across text, images, and videos. Qwen3-VL includes both dense models and Mixture-of-Experts (MoE) variants for improved efficiency. NeMo Megatron Bridge supports finetuning Qwen3-VL models (8B dense and 30B MoE variants). -```{tip} + We use the following environment variables throughout this page - `HF_MODEL_PATH=Qwen/Qwen3-VL-8B-Instruct` (or `Qwen/Qwen3-VL-30B-A3B-Instruct` for MoE) - `MEGATRON_MODEL_PATH=/models/Qwen3-VL-8B-Instruct` (feel free to set your own path) Unless explicitly stated, any megatron model path in the commands below should NOT contain the iteration number `iter_xxxxxx`. For more details on checkpointing, please see [here](https://docs.nvidia.com/nemo/megatron-bridge/latest/training/checkpointing.html#checkpoint-contents) -``` + ## Examples @@ -20,4 +25,4 @@ For checkpoint conversion, inference, finetuning recipes, and step-by-step train ## Hugging Face Model Cards - Qwen3-VL-8B: `https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct` - Qwen3-VL-30B-A3B (MoE): `https://huggingface.co/Qwen/Qwen3-VL-30B-A3B-Instruct` -- Qwen3-VL-235B-A22B (MoE): `https://huggingface.co/Qwen/Qwen3-VL-235B-A22B-Instruct` \ No newline at end of file +- Qwen3-VL-235B-A22B (MoE): `https://huggingface.co/Qwen/Qwen3-VL-235B-A22B-Instruct` diff --git a/docs/models/qwen/qwen35-vl.md b/docs/models/qwen/qwen35-vl.mdx similarity index 89% rename from docs/models/qwen/qwen35-vl.md rename to docs/models/qwen/qwen35-vl.mdx index 7b3cee0dea..b0867fbcbc 100644 --- a/docs/models/qwen/qwen35-vl.md +++ b/docs/models/qwen/qwen35-vl.mdx @@ -1,3 +1,8 @@ +--- +title: "Qwen 3.5 / 3.6" +description: "" +--- + # Qwen 3.5 / 3.6 [Qwen3.5](https://huggingface.co/collections/Qwen/qwen35) is a family of vision-language models supporting multimodal understanding across text, images, and videos. Qwen3.5-VL includes both dense models and Mixture-of-Experts (MoE) variants for improved efficiency at scale. @@ -8,9 +13,9 @@ Qwen 3.5/3.6 models feature a hybrid architecture combining GDN (Gated DeltaNet) Qwen 3.5/3.6 models are supported via Megatron Bridge with auto-detected configuration and weight mapping. -```{important} + Please upgrade to `transformers` >= 5.2.0 in order to use the Qwen 3.5 models. -``` + ## Available Models @@ -65,8 +70,8 @@ For checkpoint conversion, inference, finetuning recipes, and step-by-step train - Qwen3.6 35B-A3B (MoE): https://huggingface.co/Qwen/Qwen3.6-35B-A3B ## Related Docs -- Related Qwen variant: [Qwen3-VL](qwen3-vl.md) -- Related Qwen variant: [Qwen](qwen.md) -- Recipe usage: [Recipe usage](../../recipe-usage.md) -- Customizing the training recipe configuration: [Configuration overview](../../training/config-container-overview.md) -- Training entry points: [Entry points](../../training/entry-points.md) +- Related Qwen variant: [Qwen3-VL](/models/qwen/qwen3-vl) +- Related Qwen variant: [Qwen](/models/qwen/qwen) +- Recipe usage: [Recipe usage](/training/recipe-usage) +- Customizing the training recipe configuration: [Configuration overview](/training/config-container-overview) +- Training entry points: [Entry points](/training/entry-points) diff --git a/docs/models/sarvam/index.md b/docs/models/sarvam/index.mdx similarity index 55% rename from docs/models/sarvam/index.md rename to docs/models/sarvam/index.mdx index c13344276a..ba04282013 100644 --- a/docs/models/sarvam/index.md +++ b/docs/models/sarvam/index.mdx @@ -1,13 +1,12 @@ +--- +title: "Sarvam" +description: "" +--- + # Sarvam Sarvam model documentation is organized by model variant. -```{toctree} -:hidden: - -sarvam.md -``` - | Variant | Guide | |---------|-------| -| Sarvam | [sarvam.md](sarvam.md) | +| Sarvam | [sarvam.md](/models/sarvam/sarvam) | diff --git a/docs/models/sarvam/sarvam.md b/docs/models/sarvam/sarvam.mdx similarity index 96% rename from docs/models/sarvam/sarvam.md rename to docs/models/sarvam/sarvam.mdx index 95c13e912f..4f0b025a31 100644 --- a/docs/models/sarvam/sarvam.md +++ b/docs/models/sarvam/sarvam.mdx @@ -1,3 +1,8 @@ +--- +title: "Sarvam" +description: "" +--- + # Sarvam [Sarvam](https://huggingface.co/sarvamai/sarvam-30b) language models are MoE models from Sarvam AI. Megatron Bridge supports Sarvam dense-MLA and MoE variants through the Sarvam bridge implementations. @@ -23,4 +28,3 @@ For checkpoint import/export, round-trip validation, and inference commands, see - Bridge implementation: [`src/megatron/bridge/models/sarvam`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/src/megatron/bridge/models/sarvam) - Examples: [`examples/models/sarvam`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/models/sarvam) - diff --git a/docs/models/stepfun/index.md b/docs/models/stepfun/index.mdx similarity index 53% rename from docs/models/stepfun/index.md rename to docs/models/stepfun/index.mdx index df874834f7..c33c71bd4b 100644 --- a/docs/models/stepfun/index.md +++ b/docs/models/stepfun/index.mdx @@ -1,14 +1,12 @@ +--- +title: "StepFun" +description: "" +--- + # StepFun StepFun model documentation is organized by model variant. -```{toctree} -:hidden: - -step35.md -``` - | Variant | Guide | |---------|-------| -| Step-3.5-Flash | [step35.md](step35.md) | - +| Step-3.5-Flash | [step35.md](/models/stepfun/step35) | diff --git a/docs/models/stepfun/step35.md b/docs/models/stepfun/step35.mdx similarity index 97% rename from docs/models/stepfun/step35.md rename to docs/models/stepfun/step35.mdx index 9a4ef742a3..3449601746 100644 --- a/docs/models/stepfun/step35.md +++ b/docs/models/stepfun/step35.mdx @@ -1,3 +1,8 @@ +--- +title: "Step-3.5-Flash" +description: "" +--- + # Step-3.5-Flash [Step-3.5-Flash](https://huggingface.co/stepfun-ai/Step-3.5-Flash) is a MoE language model from StepFun. Megatron Bridge supports checkpoint conversion, inference, and continued pretraining through a dedicated Step-3.5 bridge, provider, and recipe. @@ -23,4 +28,3 @@ For conversion, inference, pretraining, Slurm launch scripts, and MCore branch n - Bridge implementation: [`src/megatron/bridge/models/stepfun`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/src/megatron/bridge/models/stepfun) - Recipe: [`src/megatron/bridge/recipes/stepfun/step35.py`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/recipes/stepfun/step35.py) - Examples: [`examples/models/stepfun/step35`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/models/stepfun/step35) - diff --git a/docs/project.json b/docs/project.json deleted file mode 100644 index 240d43ad95..0000000000 --- a/docs/project.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "name": "megatron-bridge", - "version": "nightly" -} diff --git a/docs/reference/api/index.mdx b/docs/reference/api/index.mdx new file mode 100644 index 0000000000..958f45be7d --- /dev/null +++ b/docs/reference/api/index.mdx @@ -0,0 +1,18 @@ +--- +title: "API Reference" +description: "API reference placeholder for Megatron Bridge." +--- + +# API Reference + +The previous Sphinx site generated API pages with `autodoc2` under `docs/apidocs/`. That generated content is intentionally excluded from this Fern MDX migration. + +For now, use the source tree and public package APIs directly: + +- [Megatron Bridge source](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/src/megatron/bridge) +- [Conversion APIs](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/src/megatron/bridge/models/conversion) +- [Training APIs](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/src/megatron/bridge/training) + +## Remaining Work + +Decide whether API reference pages should be hand-maintained MDX, generated static MDX, or linked externally from the Fern site. diff --git a/docs/reference/index.mdx b/docs/reference/index.mdx new file mode 100644 index 0000000000..55107a152a --- /dev/null +++ b/docs/reference/index.mdx @@ -0,0 +1,24 @@ +--- +title: "Reference" +description: "Reference documentation for APIs, performance data, and compatibility information." +--- + +# Reference + +Use reference pages when you need lookup-oriented information such as APIs, performance tables, migration mappings, or compatibility data. + + + + +Placeholder for API reference integration. Generated `docs/apidocs/` content remains excluded from this migration. + + + +Review benchmark summaries and historical performance data. + + + +Check current container, dependency, and package versions. + + + diff --git a/docs/performance-summary-archive.md b/docs/reference/performance/archive.mdx similarity index 99% rename from docs/performance-summary-archive.md rename to docs/reference/performance/archive.mdx index 3b0397dcd7..20da61747a 100644 --- a/docs/performance-summary-archive.md +++ b/docs/reference/performance/archive.mdx @@ -1,6 +1,11 @@ +--- +title: "Performance Archive" +description: "" +--- + # Performance Archive -NOTE: This page contains performance summaries for past releases. You can find the summary for the latest release at [performance-summary.md](performance-summary.md). +NOTE: This page contains performance summaries for past releases. You can find the summary for the latest release at [performance-summary.md](/reference/performance/summary). As part of the NVIDIA NeMo Framework, Megatron Bridge, provides optimal performance for training advanced generative AI models by incorporating the most recent training techniques, such as model parallelization, optimized attention mechanisms, and more, to achieve high training throughput. diff --git a/docs/reference/performance/index.mdx b/docs/reference/performance/index.mdx new file mode 100644 index 0000000000..0311d35b4c --- /dev/null +++ b/docs/reference/performance/index.mdx @@ -0,0 +1,24 @@ +--- +title: "Performance Reference" +description: "Performance benchmark and archive reference pages for Megatron Bridge." +--- + +# Performance Reference + +Use these pages for benchmark results and historical performance data. + + + + +Review current benchmark results and system comparisons. + + + +Review historical performance summaries. + + + +Tune training performance and diagnose common issues. + + + diff --git a/docs/performance-summary.md b/docs/reference/performance/summary.mdx similarity index 99% rename from docs/performance-summary.md rename to docs/reference/performance/summary.mdx index aa88edec1f..c426b36462 100644 --- a/docs/performance-summary.md +++ b/docs/reference/performance/summary.mdx @@ -1,3 +1,8 @@ +--- +title: "Performance" +description: "" +--- + # Performance As part of the NVIDIA NeMo Framework, Megatron Bridge, provides optimal performance for training advanced generative AI models by incorporating the most recent training techniques, such as model parallelization, optimized attention mechanisms, and more, to achieve high training throughput. @@ -140,4 +145,4 @@ The performance data includes: ## Archive -Performance summary for past releases can be found in the [archive](performance-summary-archive.md). +Performance summary for past releases can be found in the [archive](/reference/performance/archive). diff --git a/docs/releases/README.md b/docs/releases/README.md deleted file mode 100644 index 3d7dd06d6d..0000000000 --- a/docs/releases/README.md +++ /dev/null @@ -1,83 +0,0 @@ -# Releases - -This directory contains release information, version history, and known issues for Megatron Bridge. - -## Release Documentation - -| Document | Purpose | When to Read | -|----------|---------|--------------| -| **[Release Process](release-process.md)** | Release cycle, RC cadence, code-freeze, golden values, CI annotations | Participating in or understanding a release | -| **[Cherrypick Policy](cherrypick-policy.md)** | Rules for what may land on release branches during code-freeze and absolute freeze | Cherry-picking or reviewing during a release window | -| **[Software Versions](software-versions.md)** | Current software versions and dependencies | Checking compatibility, planning upgrades | -| **[Changelog](https://github.com/NVIDIA-NeMo/Megatron-Bridge/releases)** | Detailed release history and changes | Understanding what changed in each release | -| **[Known Issues](known-issues.md)** | Known bugs, limitations, and workarounds | Troubleshooting issues, planning workarounds | - -## Quick Navigation - -### I want to - -**๐Ÿ“‹ Check current versions** -โ†’ See [Software Versions](software-versions.md) for current versions and dependencies - -**๐Ÿ“ Review release history** -โ†’ Read [Changelog](https://github.com/NVIDIA-NeMo/Megatron-Bridge/releases) for detailed release notes - -**๐Ÿ› Find known issues** -โ†’ Check [Known Issues](known-issues.md) for bugs and workarounds - -**๐Ÿ”„ Plan an upgrade** -โ†’ Review [Changelog](https://github.com/NVIDIA-NeMo/Megatron-Bridge/releases) and [Known Issues](known-issues.md) before upgrading - -**๐Ÿ” Troubleshoot a problem** -โ†’ Check [Known Issues](known-issues.md) for reported issues and solutions - -**๐Ÿ’ Cherry-pick during a release** -โ†’ See [Cherrypick Policy](cherrypick-policy.md) for what is allowed on release branches during code-freeze and absolute freeze - -## Release Information Overview - -### Software Versions - -The [Software Versions](software-versions.md) document provides: - -- Current Megatron Bridge version -- Required dependency versions -- Compatible software versions -- Version compatibility matrix - -### Changelog - -The [Changelog](https://github.com/NVIDIA-NeMo/Megatron-Bridge/releases) includes: - -- Release dates and version numbers -- New features and enhancements -- Bug fixes and improvements -- Breaking changes and migration notes -- Deprecations and removals - -### Known Issues - -The [Known Issues](known-issues.md) document covers: - -- Known bugs and their status -- Limitations and constraints -- Workarounds and solutions -- Planned fixes and timelines - -## Related Documentation - -- **[Main Documentation](../README.md)** - Return to main documentation -- **[Documentation Guide](../documentation.md)** - Contributing to documentation - -## Staying Up to Date - -To stay informed about releases: - -1. **Check Software Versions** - Review [Software Versions](software-versions.md) regularly -2. **Read Changelog** - Review [Changelog](https://github.com/NVIDIA-NeMo/Megatron-Bridge/releases) for each release -3. **Monitor Known Issues** - Check [Known Issues](known-issues.md) for updates -4. **Follow Release Notes** - Review release announcements for important changes - ---- - -**Need version information?** Check [Software Versions](software-versions.md) or return to the [main documentation](../README.md). diff --git a/docs/resources/agent-skills.mdx b/docs/resources/agent-skills.mdx new file mode 100644 index 0000000000..f49eafe47c --- /dev/null +++ b/docs/resources/agent-skills.mdx @@ -0,0 +1,11 @@ +--- +title: "Agent Skills Reference" +description: "" +--- + +# Agent Skills Reference + +Operational guides and validated knowledge cards for Megatron Bridge. + +Each skill contains enablement snippets, code anchors, constraints, pitfalls, +and verification steps. diff --git a/docs/releases/cherrypick-policy.md b/docs/resources/contributing/cherrypick-policy.mdx similarity index 93% rename from docs/releases/cherrypick-policy.md rename to docs/resources/contributing/cherrypick-policy.mdx index 11ce71de23..6ac7ed804b 100644 --- a/docs/releases/cherrypick-policy.md +++ b/docs/resources/contributing/cherrypick-policy.mdx @@ -1,3 +1,8 @@ +--- +title: "Cherrypick Policy" +description: "" +--- + # Cherrypick Policy This policy governs what may land on release branches of @@ -6,7 +11,7 @@ This policy governs what may land on release branches of between the cut of a release branch and the corresponding release. It supplements the cadence and stabilization rules in -[Release Process](release-process.md); read that document first for the +[Release Process](/resources/contributing/release-process); read that document first for the overall release timeline. ----- @@ -139,5 +144,5 @@ team โ€” is the sole waiver of the cherrypick onto the release branch. ## See Also -- [Release Process](release-process.md) โ€” full release cadence, RC schedule, and golden-values policy. -- [Releases overview](README.md) โ€” index of all release-related documentation. +- [Release Process](/resources/contributing/release-process) โ€” full release cadence, RC schedule, and golden-values policy. +- [Releases overview](/about/release-notes) โ€” index of all release-related documentation. diff --git a/docs/resources/contributing/index.mdx b/docs/resources/contributing/index.mdx new file mode 100644 index 0000000000..e5ae7b4a8f --- /dev/null +++ b/docs/resources/contributing/index.mdx @@ -0,0 +1,24 @@ +--- +title: "Contributing Resources" +description: "Release and contribution process resources for Megatron Bridge." +--- + +# Contributing Resources + +These pages support contributors and release participants. + + + + +Review release cycle, code-freeze, and CI annotation guidance. + + + +Review release-branch cherry-pick rules. + + + +Review documentation development guidance. + + + diff --git a/docs/releases/release-process.md b/docs/resources/contributing/release-process.mdx similarity index 96% rename from docs/releases/release-process.md rename to docs/resources/contributing/release-process.mdx index 4c5c1e49e1..dddebf303a 100644 --- a/docs/releases/release-process.md +++ b/docs/resources/contributing/release-process.mdx @@ -1,3 +1,8 @@ +--- +title: "Release Developer Guide" +description: "" +--- + # Release Developer Guide ## Overview @@ -76,7 +81,7 @@ In **Week 5**, the last bulk update of golden values is performed. After that po Code-freeze lasts **two weeks** and begins when RC3 is cut. This is the **stabilization phase** โ€” no new features are landed. -> See [Cherrypick Policy](cherrypick-policy.md) for the rules on what may merge during code-freeze and the absolute freeze that follows. +> See [Cherrypick Policy](/resources/contributing/cherrypick-policy) for the rules on what may merge during code-freeze and the absolute freeze that follows. ### First Half (Weeks 3โ€“5) diff --git a/docs/resources/documentation-development.mdx b/docs/resources/documentation-development.mdx new file mode 100644 index 0000000000..b7c65785d1 --- /dev/null +++ b/docs/resources/documentation-development.mdx @@ -0,0 +1,59 @@ +--- +title: "Documentation Development" +description: "Guidance for editing Megatron Bridge documentation after the Fern MDX migration." +--- + +# Documentation Development + +Megatron Bridge documentation now uses Fern MDX. Live documentation pages are stored as `.mdx` files under `docs/`, while Fern configuration is stored under `docs/fern/`. + +## Repository Layout + +| Path | Purpose | +| --- | --- | +| `docs/**/*.mdx` | Live hand-written documentation pages. | +| `docs/fern/docs.yml` | Fern site configuration, global theme, version list, and redirects. | +| `docs/fern/fern.config.json` | Fern CLI organization and version configuration. | +| `docs/fern/versions/nightly.yml` | Nightly navigation that points back to hoisted live MDX pages. | +| `docs/images/` and `docs/training/images/` | Static image assets used by MDX pages. | + +## Edit Pages + +Use standard Markdown plus Fern MDX components such as `Note`, `Tip`, `Warning`, `Cards`, `Card`, `Tabs`, `Tab`, and `Accordion`. + +When adding a page: + +1. Place the page in the canonical section that matches its reader intent. +2. Use `.mdx` as the file extension. +3. Add frontmatter with at least `title` and `description`. +4. Link with Fern routes such as `/training/checkpointing` instead of Sphinx `.html` or `.md` paths. +5. Update `docs/fern/versions/nightly.yml` only when adding a new top-level folder or a page that should be pinned explicitly. + +## Canonical Sections + +- `about/`: Product overview, architecture, and release notes. +- `get-started/`: Prerequisites, installation, quickstarts, and migration paths. +- `models/`: Supported model-family documentation. +- `training/`: Training configuration, performance, reliability, and customization guides. +- `model-optimization/`: ModelOpt quantization, distillation, and pruning workflows. +- `use-cases/`: Procedural or integration guides that do not fit a single product area. +- `reference/`: Lookup-oriented API placeholders and performance references. +- `troubleshooting/`: Known issues and problem-resolution guidance. +- `resources/`: Documentation maps, contributing resources, and supplementary pages. + +## API Reference + +The previous Sphinx site generated API pages with `autodoc2` under `docs/apidocs/`. That generated tree is intentionally excluded from this migration. + +For now, link API mentions to [API Reference](/reference/api) or to source files on GitHub. A follow-up decision is needed for generated static MDX API pages versus hand-maintained API reference pages. + +## Local Preview + +Do not publish from a local development environment. If local preview is needed, use the Fern CLI from `docs/fern/` with the configured Fern version. + +```bash +cd docs/fern +npx -y fern-api@$(jq -r .version fern.config.json) docs dev +``` + +This migration did not run Fern build, preview, or publish commands. diff --git a/docs/resources/documentation-map.mdx b/docs/resources/documentation-map.mdx new file mode 100644 index 0000000000..53f71599df --- /dev/null +++ b/docs/resources/documentation-map.mdx @@ -0,0 +1,272 @@ +--- +title: "Megatron Bridge Documentation" +description: "" +--- + +# Megatron Bridge Documentation + +Welcome to the Megatron Bridge documentation! This guide helps you navigate our comprehensive documentation to find exactly what you need for training, converting, and working with language, vision-language, audio, and multimodal models. + +## ๐Ÿš€ Quick Start Paths + +### I want to + +**๐Ÿƒโ€โ™‚๏ธ Get started with model conversion** +-> Start with [Bridge Guide](/get-started/quickstart/hugging-face-conversion) for Hugging Face to and from Megatron conversion + +**โšก Understand parallelisms and performance** +-> Jump to [Parallelisms Guide](/training/parallelisms) and [Performance Guide](/training/performance-tuning) + +**๐Ÿš€ Start training a model** +-> See [Training Documentation](/training) for comprehensive training guides + +**๐Ÿ“š Find model documentation** +-> Browse [Supported Models](/models), organized by family and model variant + +**๐Ÿ”ง Migrate from NeMo 2 or Megatron-LM** +-> Check [NeMo 2 Migration Guide](/get-started/migration/nemo2-to-megatron-bridge) or [Megatron-LM Migration Guide](/get-started/migration/megatron-lm-to-megatron-bridge) + +**๐Ÿ“Š Use training recipes** +-> Read [Recipe Usage](/training/recipe-usage) for pre-configured training recipes + +**๐Ÿ”Œ Add support for a new model** +-> Refer to [Adding New Models](/use-cases/model-development/add-new-model) + +**๐Ÿ“‹ Check version information** +-> See [Releases Documentation](/about/release-notes) for versions, changelog, and known issues + +--- + +## ๐Ÿ‘ฅ Documentation by Role + +### For ML Engineers & Researchers + +- **Start here:** [Bridge Guide](/get-started/quickstart/hugging-face-conversion) -> [Training Documentation](/training) +- **Deep dive:** [Performance Guide](/training/performance-tuning) -> [Training Optimization Guides](/training#optimization-and-performance) +- **Model support:** [Supported Models](/models) -> [Adding New Models](/use-cases/model-development/add-new-model) + +### For Training Engineers + +- **Start here:** [Training Documentation](/training) -> [Configuration Container Overview](/training/config-container-overview) +- **Performance:** [Performance Guide](/training/performance-tuning) -> [Performance Summary](/reference/performance/summary) +- **Parallelisms:** [Parallelisms Guide](/training/parallelisms) -> [Training Optimization](/training#optimization-and-performance) + +### For Model Developers + +- **Start here:** [Bridge Guide](/get-started/quickstart/hugging-face-conversion) -> [Bridge Tech Details](/about/architecture) +- **Model support:** [Adding New Models](/use-cases/model-development/add-new-model) -> [Model Documentation](/models) +- **Integration:** [Bridge RL Integration](/use-cases/reinforcement-learning/rl-framework-integration) + +### For DevOps & Platform Teams + +- **Start here:** [Releases Documentation](/about/release-notes) -> [Software Versions](/about/release-notes/software-versions) +- **Troubleshooting:** [Known Issues](/troubleshooting/known-issues) +- **API Reference:** [API Documentation](/reference/api) + +--- + +## ๐Ÿ“š Complete Documentation Index + +### Getting Started + +| Document | Purpose | When to Read | +|----------|---------|--------------| +| **[Bridge Guide](/get-started/quickstart/hugging-face-conversion)** | Hugging Face to and from Megatron conversion guide | First time converting models | +| **[Bridge Tech Details](/about/architecture)** | Technical details of the bridge system | Understanding bridge internals | +| **[Parallelisms Guide](/training/parallelisms)** | Data and model parallelism strategies | Setting up distributed training | +| **[Performance Summary](/reference/performance/summary)** | Quick performance reference | Quick performance lookup | +| **[Performance Guide](/training/performance-tuning)** | Comprehensive performance optimization | Optimizing training performance | + +### Model Support + +| Document | Purpose | When to Read | +|----------|---------|--------------| +| **[Supported Models](/models)** | Family-organized model documentation | Working with supported model variants | +| **[Adding New Models](/use-cases/model-development/add-new-model)** | Guide for adding model support | Extending model support | + +### Training and Customization + +| Document | Purpose | When to Read | +|----------|---------|--------------| +| **[Training Documentation](/training)** | Comprehensive training guides | Setting up and customizing training | +| **[Configuration Container Overview](/training/config-container-overview)** | Central training configuration | Understanding training configuration | +| **[Entry Points](/training/entry-points)** | Training entry points and execution | Understanding training flow | +| **[Training Loop Settings](/training/training-loop-settings)** | Training loop parameters | Configuring training parameters | +| **[Optimizer & Scheduler](/training/optimizer-scheduler)** | Optimization configuration | Setting up optimizers | +| **[Mixed Precision](/training/mixed-precision)** | Mixed precision training | Reducing memory usage | +| **[PEFT](/training/peft)** | Parameter-efficient fine-tuning | Fine-tuning with limited resources | +| **[Checkpointing](/training/checkpointing)** | Checkpoint management | Saving and resuming training | +| **[Logging](/training/logging)** | Logging and monitoring | Monitoring training progress | +| **[Profiling](/training/profiling)** | Performance profiling | Identifying bottlenecks | + +### Recipes and Workflows + +| Document | Purpose | When to Read | +|----------|---------|--------------| +| **[Recipe Usage](/training/recipe-usage)** | Using pre-configured training recipes | Quick training setup | +| **[Bridge RL Integration](/use-cases/reinforcement-learning/rl-framework-integration)** | Reinforcement learning integration | RL training workflows | + +### Migration Guides + +| Document | Purpose | When to Read | +|----------|---------|--------------| +| **[NeMo 2 Migration Guide](/get-started/migration/nemo2-to-megatron-bridge)** | Migrating from NeMo 2 | Upgrading from NeMo 2 | +| **[Megatron-LM Migration Guide](/get-started/migration/megatron-lm-to-megatron-bridge)** | Migrating from Megatron-LM | Upgrading from Megatron-LM | + +### Reference + +| Document | Purpose | When to Read | +|----------|---------|--------------| +| **[API Documentation](/reference/api)** | Complete API reference | Building integrations | +| **[Releases Documentation](/about/release-notes)** | Version history and known issues | Checking versions, troubleshooting | +| **[Documentation Guide](/resources/documentation-development)** | Contributing to documentation | Contributing docs | + +--- + +## ๐Ÿ—บ๏ธ Common Reading Paths + +### ๐Ÿ†• First-Time Users + +1. [Bridge Guide](/get-started/quickstart/hugging-face-conversion) *(10 min - understand conversion)* +2. [Parallelisms Guide](/training/parallelisms) *(15 min - understand distributed training)* +3. [Training Documentation](/training) *(choose your training path)* +4. [Recipe Usage](/training/recipe-usage) *(5 min - use pre-configured recipes)* + +### ๐Ÿ”ง Setting Up Training + +1. [Training Documentation](/training) *(overview of training system)* +2. [Configuration Container Overview](/training/config-container-overview) *(understand configuration)* +3. [Entry Points](/training/entry-points) *(how training starts)* +4. [Training Loop Settings](/training/training-loop-settings) *(configure parameters)* +5. [Logging](/training/logging) *(set up monitoring)* + +### โšก Performance Optimization + +1. [Performance Guide](/training/performance-tuning) *(comprehensive optimization strategies)* +2. [Performance Summary](/reference/performance/summary) *(quick reference)* +3. [Mixed Precision](/training/mixed-precision) *(reduce memory usage)* +4. [Communication Overlap](/training/communication-overlap) *(optimize distributed training)* +5. [Activation Recomputation](/training/activation-recomputation) *(reduce memory footprint)* +6. [Profiling](/training/profiling) *(identify bottlenecks)* + +### ๐Ÿ”„ Model Conversion Workflow + +1. [Bridge Guide](/get-started/quickstart/hugging-face-conversion) *(conversion basics)* +2. [Bridge Tech Details](/about/architecture) *(technical details)* +3. [Supported Models](/models) *(model-specific guides)* +4. [Adding New Models](/use-cases/model-development/add-new-model) *(extend support)* + +### ๐Ÿ”ง Customization and Extension + +1. [Training Documentation](/training) *(training customization)* +2. [PEFT](/training/peft) *(parameter-efficient fine-tuning)* +3. [Adding New Models](/use-cases/model-development/add-new-model) *(add model support)* +4. [Bridge RL Integration](/use-cases/reinforcement-learning/rl-framework-integration) *(RL workflows)* + +### ๐Ÿช„ Model Optimization (ModelOpt) + +1. [Model Optimization Overview](/model-optimization) *(quantization, distillation, pruning)* +2. [Quantization](/model-optimization/quantization) *(PTQ and QAT)* +3. [Distillation](/model-optimization/distillation) *(knowledge distillation)* +4. [Pruning](/model-optimization/pruning) *(Minitron-based pruning)* + +### ๐Ÿ“ฆ Migration Paths + +1. [NeMo 2 Migration Guide](/get-started/migration/nemo2-to-megatron-bridge) *(from NeMo 2)* +2. [Megatron-LM Migration Guide](/get-started/migration/megatron-lm-to-megatron-bridge) *(from Megatron-LM)* +3. [Training Documentation](/training) *(new training system)* + +--- + +## ๐Ÿ“ Directory Structure + +### Main Documentation + +- **Guides** - Core guides for parallelisms, performance, recipes, and migration +- **Bridge Documentation** - Hugging Face to and from Megatron conversion guides +- **Model Documentation** - Supported model families and architectures + +### Subdirectories + +#### [models/](/models) + +- Family folders such as **[qwen/](/models/qwen)**, **[gemma/](/models/gemma)**, and **[nemotron/](/models/nemotron)** + - Model-variant guides for conversion, training recipes, and architecture notes + - Multimodal variants live with their model families + +#### [training/](/training) + +- **Configuration** - ConfigContainer, entry points, training loop settings +- **Optimization** - Optimizer, scheduler, mixed precision, communication overlap +- **Performance** - Attention optimizations, activation recomputation, CPU offloading +- **Monitoring** - Logging, profiling, checkpointing, resiliency +- **Advanced** - PEFT, packed sequences + +#### [modelopt/](/model-optimization) + +- **Quantization** - PTQ and QAT workflows (FP8, INT8, INT4) +- **Distillation** - Knowledge distillation from teacher to student +- **Pruning** - Minitron-based depth and width pruning + +#### [releases/](/about/release-notes) + +- **Software Versions** - Current versions and dependencies +- **Changelog** - Release history and changes +- **Known Issues** - Bugs, limitations, and workarounds + +--- + +## ๐Ÿ”— How Documents Connect + +```mermaid +graph TD + A[README.md
Start Here] --> B[Bridge Guide
Model Conversion] + A --> C[Training Docs
Training Setup] + A --> D[Models
Model Support] + + B --> E[Bridge Tech Details
Technical Deep Dive] + B --> F[Supported Models
Model-Specific Guides] + + C --> G[Config Container
Configuration] + C --> H[Performance Guide
Optimization] + C --> I[Parallelisms
Distributed Training] + + G --> J[Training Loop
Training Parameters] + G --> K[Optimizer & Scheduler
Optimization Setup] + + H --> L[Mixed Precision
Memory Efficiency] + H --> M[Communication Overlap
Performance] + + I --> N[Data Parallelism
DDP] + I --> O[Model Parallelism
TP/PP/VPP] + + D --> P[Family Index
Model Variants] + D --> Q[Model Pages
Recipes and Conversion] + + style A fill:#e1f5fe + style B fill:#f3e5f5 + style C fill:#e8f5e8 + style D fill:#fff3e0 + style H fill:#fce4ec + style I fill:#e0f2f1 +``` + +--- + +## ๐Ÿค Getting Help + +- **GitHub Issues:** [Report bugs or request features](https://github.com/NVIDIA-NeMo/Megatron-Bridge/issues) +- **Documentation Issues:** Found something unclear? Let us know! +- **Community:** Join discussions and share experiences + +--- + +## ๐Ÿ“– Additional Resources + +- **[Examples](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples)** - Code examples and tutorials +- **[Contributing Guide](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/CONTRIBUTING.md)** - How to contribute to the project +- **[API Documentation](/reference/api)** - Complete API reference + +--- + +**Ready to get started?** Choose your path above or dive into the [Bridge Guide](/get-started/quickstart/hugging-face-conversion) for model conversion! ๐Ÿš€ diff --git a/docs/resources/index.mdx b/docs/resources/index.mdx new file mode 100644 index 0000000000..cb5e4aef51 --- /dev/null +++ b/docs/resources/index.mdx @@ -0,0 +1,28 @@ +--- +title: "Resources" +description: "Supplementary resources for Megatron Bridge documentation, contributing, and agent skills." +--- + +# Resources + +Use these pages for documentation maps, contribution-oriented guides, and supplementary project resources. + + + + +Browse the converted documentation index from the previous Sphinx site. + + + +Review documentation authoring guidance. This page still contains Sphinx-era details that need a Fern-specific rewrite. + + + +Review release process and cherry-pick policy pages. + + + +Review project skill documentation links. + + + diff --git a/docs/skills-index.md b/docs/skills-index.md deleted file mode 100644 index 828d1ec3bd..0000000000 --- a/docs/skills-index.md +++ /dev/null @@ -1,69 +0,0 @@ -# Agent Skills Reference - -Operational guides and validated knowledge cards for Megatron Bridge. - -Each skill contains enablement snippets, code anchors, constraints, pitfalls, -and verification steps. - -```{toctree} -:caption: Getting Started -:maxdepth: 1 - -skills/build-and-dependency/SKILL -skills/bump-dependency/SKILL -skills/testing/SKILL -skills/cicd/SKILL -skills/review-pr/SKILL -skills/nemo-mbridge-mlm-bridge-training/SKILL -skills/nemo-mbridge-recipe-recommender/SKILL -``` - -```{toctree} -:caption: Adding Model Support -:maxdepth: 1 - -skills/adding-model-support/SKILL -skills/adding-model-support/llm-patterns -skills/adding-model-support/vlm-patterns -skills/adding-model-support/recipe-patterns -skills/adding-model-support/tests-and-examples -skills/verl-e2e-testing/SKILL -skills/nemo-rl-e2e-testing/SKILL -``` - -```{toctree} -:caption: Performance Techniques -:maxdepth: 1 - -skills/nemo-mbridge-perf-cpu-offloading/SKILL -skills/nemo-mbridge-perf-moe-long-context/SKILL -skills/nemo-mbridge-perf-moe-optimization-workflow/SKILL -skills/nemo-mbridge-perf-moe-vlm-training/SKILL -skills/nemo-mbridge-perf-memory-tuning/SKILL -skills/nemo-mbridge-perf-moe-dispatcher-selection/SKILL -skills/nemo-mbridge-perf-moe-hardware-configs/SKILL -skills/nemo-mbridge-perf-parallelism-strategies/SKILL -skills/nemo-mbridge-perf-cuda-graphs/SKILL -skills/nemo-mbridge-perf-tp-dp-comm-overlap/SKILL -skills/nemo-mbridge-perf-megatron-fsdp/SKILL -skills/nemo-mbridge-perf-sequence-packing/SKILL -skills/nemo-mbridge-perf-hierarchical-context-parallel/SKILL -skills/nemo-mbridge-perf-expert-parallel-overlap/SKILL -skills/nemo-mbridge-perf-moe-comm-overlap/SKILL -skills/nemo-mbridge-perf-activation-recompute/SKILL -``` - -```{toctree} -:caption: Cluster & Debugging -:maxdepth: 1 - -skills/nemo-mbridge-multi-node-slurm/SKILL -skills/nemo-mbridge-multi-node-slurm/references/templates -``` - -```{toctree} -:caption: Resiliency -:maxdepth: 1 - -skills/nemo-mbridge-resiliency/SKILL -``` diff --git a/docs/training/README.md b/docs/training/README.md deleted file mode 100644 index 9f16c45330..0000000000 --- a/docs/training/README.md +++ /dev/null @@ -1,126 +0,0 @@ -# Training and Customization - -This directory contains comprehensive documentation for training and customizing models with Megatron Bridge. Learn how to configure training, optimize performance, and customize training workflows. - -## Quick Navigation - -### I want to - -**๐Ÿš€ Get started with training** -โ†’ Start with [Configuration Container Overview](config-container-overview.md) and [Data Preparation](data-preparation.md) to understand the training setup - -**โš™๏ธ Configure training parameters** -โ†’ See [Training Loop Settings](training-loop-settings.md) and [Optimizer & Scheduler](optimizer-scheduler.md) - -**๐Ÿ“Š Monitor and profile training** -โ†’ Check [Logging](logging.md) and [Profiling](profiling.md) guides - -**๐Ÿ’พ Manage checkpoints** -โ†’ Read [Checkpointing](checkpointing.md) for saving and resuming training - -**โšก Optimize performance** -โ†’ Explore [Performance Guide](../performance-guide.md) and [Performance Summary](../performance-summary.md) - -**๐Ÿ”ง Customize training** -โ†’ See [PEFT](peft.md), [Distillation](../modelopt/distillation.md), [Entry Points](entry-points.md), and [Callbacks](callbacks.md) - -## Core Training Documentation - -### Configuration and Setup - -| Document | Purpose | When to Read | -|----------|---------|--------------| -| **[Configuration Container Overview](config-container-overview.md)** | Central configuration object for all training settings | First time setting up training | -| **[Entry Points](entry-points.md)** | Training entry points and execution flow | Understanding how training starts | -| **[Data Preparation](data-preparation.md)** | Dataset formats for pretraining, SFT, PEFT, and VLM fine-tuning | Preparing data or choosing dataset config fields | -| **[Training Loop Settings](training-loop-settings.md)** | Training loop parameters and configuration | Configuring batch sizes, iterations, validation | - -### Optimization and Performance - -| Document | Purpose | When to Read | -|----------|---------|--------------| -| **[Optimizer & Scheduler](optimizer-scheduler.md)** | Optimizer and learning rate scheduler configuration | Setting up optimization | -| **[Mixed Precision](mixed-precision.md)** | Mixed precision training for memory efficiency | Reducing memory usage | -| **[Communication Overlap](communication-overlap.md)** | Overlapping communication with computation | Optimizing distributed training | -| **[Hierarchical Context Parallel](hierarchical-context-parallel.md)** | Hierarchical `a2a+p2p` context parallel guidance | Advanced long-sequence scaling | -| **[Attention Optimizations](attention-optimizations.md)** | Optimizing attention mechanisms | Improving training speed | -| **[Activation Recomputation](activation-recomputation.md)** | Gradient checkpointing strategies | Reducing memory footprint | -| **[CPU Offloading](cpu-offloading.md)** | Offloading to CPU for memory management | Working with limited GPU memory | - -### Monitoring and Debugging - -| Document | Purpose | When to Read | -|----------|---------|--------------| -| **[Logging](logging.md)** | Logging configuration and TensorBoard/WandB integration | Monitoring training progress | -| **[Profiling](profiling.md)** | Performance profiling and analysis | Identifying bottlenecks | -| **[Theoretical Memory Estimator](memory-estimator.md)** | Formula-based per-GPU memory planning | Sizing dense and MoE training configs | -| **[Resiliency](resiliency.md)** | Handling failures and recovery | Building robust training pipelines | - -### Advanced Features - -| Document | Purpose | When to Read | -|----------|---------|--------------| -| **[PEFT](peft.md)** | Parameter-Efficient Fine-Tuning (LoRA, etc.) | Fine-tuning with limited resources | -| **[Packed Sequences](packed-sequences.md)** | Sequence packing for efficiency | Optimizing data loading | -| **[Megatron FSDP](megatron-fsdp.md)** | Stable overview of Megatron FSDP | Choosing an FSDP path | -| **[Distillation](../modelopt/distillation.md)** | Knowledge distillation techniques | Transferring knowledge between models | -| **[Checkpointing](checkpointing.md)** | Checkpoint saving, loading, and resuming | Managing training state | -| **[Callbacks](callbacks.md)** | Inject custom logic into training loop | Custom logging, metrics, third-party integrations | - -## Training Workflow - -A typical training workflow involves: - -1. **Configure Training** - Set up `ConfigContainer` with model, data, and training parameters -2. **Prepare Data** - Configure dataset loading and preprocessing with the right data format -3. **Set Optimization** - Configure optimizer, scheduler, and mixed precision -4. **Enable Monitoring** - Set up logging and profiling -5. **Configure Checkpointing** - Set up checkpoint saving and resuming -6. **Launch Training** - Start training with configured entry points -7. **Monitor Progress** - Track metrics via logging and profiling -8. **Resume if Needed** - Use checkpointing to resume from saved state - -## Related Documentation - -- **[Main Documentation Index](../index.md)** - Return to main documentation -- **[Performance Guide](../performance-guide.md)** - Comprehensive performance optimization guide -- **[Performance Summary](../performance-summary.md)** - Quick performance reference -- **[Recipe Usage](../recipe-usage.md)** - Using training recipes -- **[Parallelisms](../parallelisms.md)** - Understanding distributed training strategies -- **[Bridge Guide](../bridge-guide.md)** - Working with Hugging Face models - -## Common Training Scenarios - -### ๐Ÿ†• First-Time Training Setup - -1. [Configuration Container Overview](config-container-overview.md) - Understand the configuration system -2. [Data Preparation](data-preparation.md) - Choose the right dataset format and config fields -3. [Entry Points](entry-points.md) - Learn how to start training -4. [Training Loop Settings](training-loop-settings.md) - Configure basic training parameters -5. [Logging](logging.md) - Set up monitoring - -### โšก Performance Optimization - -1. [Performance Guide](../performance-guide.md) - Comprehensive optimization strategies -2. [Mixed Precision](mixed-precision.md) - Enable mixed precision training -3. [Communication Overlap](communication-overlap.md) - Optimize distributed training -4. [Activation Recomputation](activation-recomputation.md) - Reduce memory usage -5. [Profiling](profiling.md) - Identify bottlenecks - -### ๐Ÿ’พ Production Training - -1. [Checkpointing](checkpointing.md) - Reliable checkpoint management -2. [Resiliency](resiliency.md) - Handle failures gracefully -3. [Logging](logging.md) - Comprehensive monitoring -4. [Profiling](profiling.md) - Performance analysis - -### ๐Ÿ”ง Customization - -1. [PEFT](peft.md) - Parameter-efficient fine-tuning -2. [Distillation](../modelopt/distillation.md) - Knowledge distillation -3. [Entry Points](entry-points.md) - Custom training workflows -4. [Callbacks](callbacks.md) - Inject custom logic (third-party integrations) - ---- - -**Ready to start training?** Begin with [Configuration Container Overview](config-container-overview.md) or return to the [main documentation](../README.md). diff --git a/docs/training/activation-recomputation.md b/docs/training/activation-recomputation.mdx similarity index 91% rename from docs/training/activation-recomputation.md rename to docs/training/activation-recomputation.mdx index 0e1d47b5ed..f9fa016925 100644 --- a/docs/training/activation-recomputation.md +++ b/docs/training/activation-recomputation.mdx @@ -1,3 +1,8 @@ +--- +title: "Activation Recomputation" +description: "" +--- + # Activation Recomputation The input activations of network layers are stored in device memory and are used @@ -191,7 +196,7 @@ up-projection modules than to enable blanket full recomputation immediately. ## Related Docs -- [docs/training/cuda-graphs.md](cuda-graphs.md) -- [docs/training/moe-optimization.md](moe-optimization.md) -- [skills/nemo-mbridge-perf-activation-recompute/SKILL.md](../skills/nemo-mbridge-perf-activation-recompute/SKILL.md) โ€” per-module cost/savings data, measured results -- [skills/nemo-mbridge-perf-memory-tuning/SKILL.md](../skills/nemo-mbridge-perf-memory-tuning/SKILL.md) โ€” expandable segments, parallelism resizing, and other memory reduction strategies +- [docs/training/cuda-graphs.mdx](/training/cuda-graphs) +- [docs/training/moe-optimization.mdx](/training/moe-optimization) +- [skills/nemo-mbridge-perf-activation-recompute/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-activation-recompute/SKILL.md) โ€” per-module cost/savings data, measured results +- [skills/nemo-mbridge-perf-memory-tuning/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-memory-tuning/SKILL.md) โ€” expandable segments, parallelism resizing, and other memory reduction strategies diff --git a/docs/training/attention-optimizations.md b/docs/training/attention-optimizations.mdx similarity index 99% rename from docs/training/attention-optimizations.md rename to docs/training/attention-optimizations.mdx index 74cc82a9e4..ea8faadfc1 100644 --- a/docs/training/attention-optimizations.md +++ b/docs/training/attention-optimizations.mdx @@ -1,3 +1,8 @@ +--- +title: "Attention Optimizations" +description: "" +--- + # Attention Optimizations Megatron Bridge provides several attention optimizations to improve the efficiency and performance of transformer models. These optimizations include Flash Attention for memory efficiency, and Multi-Query Attention (MQA) and Grouped-Query Attention (GQA) for computational efficiency. diff --git a/docs/training/callbacks.md b/docs/training/callbacks.mdx similarity index 92% rename from docs/training/callbacks.md rename to docs/training/callbacks.mdx index 15d619247e..95416f9544 100644 --- a/docs/training/callbacks.md +++ b/docs/training/callbacks.mdx @@ -1,3 +1,8 @@ +--- +title: "Callbacks" +description: "" +--- + # Callbacks Megatron Bridge provides a lightweight callback system for injecting custom logic into the training and evaluation loop without modifying framework code. This is ideal for propietary integrations or custom logging and metrics tracking. @@ -6,7 +11,7 @@ Megatron Bridge provides a lightweight callback system for injecting custom logi ### Class-Based Callbacks -Subclass {py:class}`bridge.training.callbacks.Callback` and override event methods: +Subclass `bridge.training.callbacks.Callback` and override event methods: ```python import time @@ -38,7 +43,7 @@ pretrain(config, forward_step, callbacks=[MyCallback()]) ### Functional Callbacks -Register functions directly with {py:class}`bridge.training.callbacks.CallbackManager`: +Register functions directly with `bridge.training.callbacks.CallbackManager`: ```python from megatron.bridge.training.callbacks import CallbackManager @@ -118,11 +123,11 @@ pretrain(config, forward_step, callbacks=manager) ## CallbackContext -The {py:class}`bridge.training.callbacks.CallbackContext` provides access to framework state: +The `bridge.training.callbacks.CallbackContext` provides access to framework state: ### Always Available -- **`state`**: {py:class}`bridge.training.state.GlobalState` - Contains config, train_state, timers, and loggers +- **`state`**: `bridge.training.state.GlobalState` - Contains config, train_state, timers, and loggers - **`model`**: List of model chunks - **`user_state`**: Mutable dict for storing data across callback invocations @@ -242,8 +247,8 @@ class ProprietaryMetricsCallback(Callback): ## API Reference -- {py:class}`bridge.training.callbacks.Callback` -- {py:class}`bridge.training.callbacks.CallbackContext` -- {py:class}`bridge.training.callbacks.CallbackManager` -- {py:func}`bridge.training.callbacks.normalize_callbacks` -- {py:func}`bridge.training.callbacks.should_fire` +- `bridge.training.callbacks.Callback` +- `bridge.training.callbacks.CallbackContext` +- `bridge.training.callbacks.CallbackManager` +- `bridge.training.callbacks.normalize_callbacks` +- `bridge.training.callbacks.should_fire` diff --git a/docs/training/checkpointing.md b/docs/training/checkpointing.mdx similarity index 96% rename from docs/training/checkpointing.md rename to docs/training/checkpointing.mdx index 8ee479f269..2858faedc0 100644 --- a/docs/training/checkpointing.md +++ b/docs/training/checkpointing.mdx @@ -1,10 +1,15 @@ +--- +title: "Checkpointing" +description: "" +--- + # Checkpointing -The {py:class}`bridge.training.config.CheckpointConfig` controls model checkpointing behavior, including saving and loading checkpoints, checkpoint formats, and various optimization features. +The `bridge.training.config.CheckpointConfig` controls model checkpointing behavior, including saving and loading checkpoints, checkpoint formats, and various optimization features. -```{Note} -This documentation covers **Megatron-format checkpoints** used during training. For converting between ๐Ÿค— Hugging Face and Megatron formats, see the {doc}`../bridge-guide`. -``` + +This documentation covers **Megatron-format checkpoints** used during training. For converting between ๐Ÿค— Hugging Face and Megatron formats, see the [bridge guide](/get-started/quickstart/hugging-face-conversion). + ## Overview @@ -76,11 +81,11 @@ checkpoint = CheckpointConfig( ) ``` -```{note} + The `load` parameter should always point to the base checkpoint directory (not the `iter_N` subdirectory). The `ckpt_step` parameter overrides which iteration is loaded from that directory. **Important:** If `ckpt_step` is specified but the checkpoint directory does not exist, training will **fail immediately** with a `FileNotFoundError`. This is intentional to prevent accidentally starting training from scratch when you meant to resume from a specific checkpoint. -``` + ### Default Recipe Resume Behavior @@ -185,7 +190,7 @@ checkpoint = CheckpointConfig( | `zarr` | โœ… | โœ… | โŒ | โœ… | โŒ | | `fsdp_dtensor` | โŒ | โŒ | โœ… | โŒ | โŒ | -**Important**: When using Megatron FSDP (`use_megatron_fsdp=True`), you must set `ckpt_format="fsdp_dtensor"`. Other formats are not compatible with FSDP's sharded parameter layout. See {doc}`megatron-fsdp` for complete FSDP configuration details. +**Important**: When using Megatron FSDP (`use_megatron_fsdp=True`), you must set `ckpt_format="fsdp_dtensor"`. Other formats are not compatible with FSDP's sharded parameter layout. See [megatron fsdp](/training/megatron-fsdp) for complete FSDP configuration details. ## Performance Optimizations @@ -445,6 +450,6 @@ When `custom_manager_class` is not set, Megatron Bridge uses `DefaultCheckpointM ## Related Documentation -- {doc}`megatron-fsdp` - Megatron FSDP configuration and `fsdp_dtensor` format requirements -- {doc}`../parallelisms` - Understanding data and model parallelism strategies -- {doc}`config-container-overview` - Complete configuration reference +- [megatron fsdp](/training/megatron-fsdp) - Megatron FSDP configuration and `fsdp_dtensor` format requirements +- [parallelisms](/training/parallelisms) - Understanding data and model parallelism strategies +- [config container overview](/training/config-container-overview) - Complete configuration reference diff --git a/docs/training/communication-overlap.md b/docs/training/communication-overlap.mdx similarity index 83% rename from docs/training/communication-overlap.md rename to docs/training/communication-overlap.mdx index bb4b8bf371..592a3cada9 100644 --- a/docs/training/communication-overlap.md +++ b/docs/training/communication-overlap.mdx @@ -1,3 +1,8 @@ +--- +title: "Communication Overlap" +description: "" +--- + # Communication Overlap Communication overlap reduces exposed communication cost in distributed training @@ -7,8 +12,8 @@ This page is the stable guide for what communication overlap is, when it tends to help, and which boundaries are durable across Megatron Bridge. For exact knobs, code anchors, and verification commands, see: -- [skills/nemo-mbridge-perf-expert-parallel-overlap/SKILL.md](../skills/nemo-mbridge-perf-expert-parallel-overlap/SKILL.md) -- [skills/nemo-mbridge-perf-tp-dp-comm-overlap/SKILL.md](../skills/nemo-mbridge-perf-tp-dp-comm-overlap/SKILL.md) +- [skills/nemo-mbridge-perf-expert-parallel-overlap/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-expert-parallel-overlap/SKILL.md) +- [skills/nemo-mbridge-perf-tp-dp-comm-overlap/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-tp-dp-comm-overlap/SKILL.md) ## What It Is @@ -122,8 +127,8 @@ on the actual bottleneck. For config examples and minimal runnable commands, see: -- [skills/nemo-mbridge-perf-expert-parallel-overlap/SKILL.md](../skills/nemo-mbridge-perf-expert-parallel-overlap/SKILL.md) -- [skills/nemo-mbridge-perf-tp-dp-comm-overlap/SKILL.md](../skills/nemo-mbridge-perf-tp-dp-comm-overlap/SKILL.md) +- [skills/nemo-mbridge-perf-expert-parallel-overlap/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-expert-parallel-overlap/SKILL.md) +- [skills/nemo-mbridge-perf-tp-dp-comm-overlap/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-tp-dp-comm-overlap/SKILL.md) ## Expected Metric Changes @@ -155,11 +160,10 @@ correct question is always "which communication path is exposed in this run?" ## Related Docs -- [docs/performance-guide.md](../performance-guide.md) -- [docs/training/cuda-graphs.md](cuda-graphs.md) -- [docs/training/hierarchical-context-parallel.md](hierarchical-context-parallel.md) -- [skills/nemo-mbridge-perf-expert-parallel-overlap/SKILL.md](../skills/nemo-mbridge-perf-expert-parallel-overlap/SKILL.md) -- [skills/nemo-mbridge-perf-moe-comm-overlap/SKILL.md](../skills/nemo-mbridge-perf-moe-comm-overlap/SKILL.md) -- [skills/nemo-mbridge-perf-moe-comm-overlap/card.yaml](../skills/nemo-mbridge-perf-moe-comm-overlap/card.yaml) -- [skills/nemo-mbridge-perf-tp-dp-comm-overlap/SKILL.md](../skills/nemo-mbridge-perf-tp-dp-comm-overlap/SKILL.md) - +- [docs/training/performance-tuning.mdx](/training/performance-tuning) +- [docs/training/cuda-graphs.mdx](/training/cuda-graphs) +- [docs/training/hierarchical-context-parallel.mdx](/training/hierarchical-context-parallel) +- [skills/nemo-mbridge-perf-expert-parallel-overlap/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-expert-parallel-overlap/SKILL.md) +- [skills/nemo-mbridge-perf-moe-comm-overlap/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-moe-comm-overlap/SKILL.md) +- [skills/nemo-mbridge-perf-moe-comm-overlap/card.yaml](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-moe-comm-overlap/card.yaml) +- [skills/nemo-mbridge-perf-tp-dp-comm-overlap/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-tp-dp-comm-overlap/SKILL.md) diff --git a/docs/training/config-container-overview.md b/docs/training/config-container-overview.mdx similarity index 99% rename from docs/training/config-container-overview.md rename to docs/training/config-container-overview.mdx index 3a003bd326..242f64064a 100644 --- a/docs/training/config-container-overview.md +++ b/docs/training/config-container-overview.mdx @@ -1,3 +1,8 @@ +--- +title: "Configuration Overview" +description: "" +--- + # Configuration Overview The `ConfigContainer` is the central configuration object in Megatron Bridge that holds all settings for training. It acts as a single source of truth that brings together model architecture, training parameters, data loading, optimization, checkpointing, logging, and distributed training settings. diff --git a/docs/training/cpu-offloading.md b/docs/training/cpu-offloading.mdx similarity index 88% rename from docs/training/cpu-offloading.md rename to docs/training/cpu-offloading.mdx index 56a8a239ff..cfc50839a5 100644 --- a/docs/training/cpu-offloading.md +++ b/docs/training/cpu-offloading.mdx @@ -1,3 +1,8 @@ +--- +title: "CPU Offloading" +description: "" +--- + # CPU Offloading CPU offloading reduces per-GPU memory by moving data to host (CPU) memory @@ -5,7 +10,7 @@ during training, trading throughput for the ability to train models or configurations that would otherwise not fit in GPU memory. For operational setup, code anchors, and verification commands, see -[skills/nemo-mbridge-perf-cpu-offloading/SKILL.md](../skills/nemo-mbridge-perf-cpu-offloading/SKILL.md). +[skills/nemo-mbridge-perf-cpu-offloading/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-cpu-offloading/SKILL.md). ## What It Is @@ -96,7 +101,7 @@ CPU offloading is configured through two independent config namespaces: `model.cpu_offloading_num_layers`, and related `model.cpu_offloading_*` fields For config examples, parameter tables, and runnable commands, see -[skills/nemo-mbridge-perf-cpu-offloading/SKILL.md](../skills/nemo-mbridge-perf-cpu-offloading/SKILL.md). +[skills/nemo-mbridge-perf-cpu-offloading/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-cpu-offloading/SKILL.md). ## Common Failure Modes @@ -111,8 +116,8 @@ For config examples, parameter tables, and runnable commands, see ## Related Docs -- [docs/training/activation-recomputation.md](activation-recomputation.md) -- [docs/training/megatron-fsdp.md](megatron-fsdp.md) -- [docs/training/optimizer-scheduler.md](optimizer-scheduler.md) -- [docs/training/cuda-graphs.md](cuda-graphs.md) -- [skills/nemo-mbridge-perf-cpu-offloading/SKILL.md](../skills/nemo-mbridge-perf-cpu-offloading/SKILL.md) +- [docs/training/activation-recomputation.mdx](/training/activation-recomputation) +- [docs/training/megatron-fsdp.mdx](/training/megatron-fsdp) +- [docs/training/optimizer-scheduler.mdx](/training/optimizer-scheduler) +- [docs/training/cuda-graphs.mdx](/training/cuda-graphs) +- [skills/nemo-mbridge-perf-cpu-offloading/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-cpu-offloading/SKILL.md) diff --git a/docs/training/cuda-graphs.md b/docs/training/cuda-graphs.mdx similarity index 93% rename from docs/training/cuda-graphs.md rename to docs/training/cuda-graphs.mdx index 4438ef0da7..e21f581a4e 100644 --- a/docs/training/cuda-graphs.md +++ b/docs/training/cuda-graphs.mdx @@ -1,3 +1,8 @@ +--- +title: "CUDA Graphs" +description: "" +--- + # CUDA Graphs CUDA graphs capture a sequence of GPU operations once and replay them with @@ -7,7 +12,7 @@ every training step. This page is the stable guide for what CUDA graphs are, when they help, and what tradeoffs to expect. For exact enablement knobs, code anchors, and verification commands, see -[skills/nemo-mbridge-perf-cuda-graphs/SKILL.md](../skills/nemo-mbridge-perf-cuda-graphs/SKILL.md). +[skills/nemo-mbridge-perf-cuda-graphs/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-cuda-graphs/SKILL.md). ## What It Is @@ -107,7 +112,7 @@ If you choose `local` with `full_iteration`, disable the loss and gradient NaN checks that conflict with full capture. For exact config snippets and runnable commands, see -[skills/nemo-mbridge-perf-cuda-graphs/SKILL.md](../skills/nemo-mbridge-perf-cuda-graphs/SKILL.md). +[skills/nemo-mbridge-perf-cuda-graphs/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-cuda-graphs/SKILL.md). ## Minimal Runnable Example @@ -194,6 +199,6 @@ Larger MoE runs can become memory-gated before graph replay pays off: ## Related Docs -- [Performance Guide](../performance-guide.md) -- [Communication Overlap](communication-overlap.md) -- [skills/nemo-mbridge-perf-cuda-graphs/SKILL.md](../skills/nemo-mbridge-perf-cuda-graphs/SKILL.md) +- [Performance Guide](/training/performance-tuning) +- [Communication Overlap](/training/communication-overlap) +- [skills/nemo-mbridge-perf-cuda-graphs/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-cuda-graphs/SKILL.md) diff --git a/docs/training/data-preparation.mdx b/docs/training/data-preparation.mdx new file mode 100644 index 0000000000..f1c3a089ee --- /dev/null +++ b/docs/training/data-preparation.mdx @@ -0,0 +1,168 @@ +--- +title: "Data Preparation" +description: "" +--- + +# Data Preparation + +Megatron Bridge uses different dataset config objects for pretraining, text fine-tuning, and multimodal fine-tuning. Choose the data path by workflow first, then keep the dataset sequence length aligned with `model.seq_length`. + +## Data Formats by Workflow + +| Workflow | Data format | Config or provider | Required path fields | +|----------|-------------|--------------------|----------------------| +| LLM pretraining | Megatron binary `.bin`/`.idx` prefixes | `GPTDatasetConfig` | `data_path`, `blend`, or `blend_per_split` | +| LLM SFT or PEFT from local files | JSONL split files | `FinetuningDatasetConfig` | `dataset_root` | +| LLM SFT or PEFT from Hugging Face datasets | Hugging Face dataset processed to JSONL | `HFDatasetConfig` | `dataset_name`, `process_example_fn`, optional `dataset_root` | +| VLM SFT or PEFT | Energon/WebDataset, Hugging Face VLM dataset, or preloaded JSON | VLM `DatasetProvider` | Provider-specific fields such as `path`, `train_data_path`, or `image_folder` | + +Use `seq_length` in Bridge examples and CLI overrides. `GPTDatasetConfig` also stores this value as Megatron Core's inherited `sequence_length` field internally, but `FinetuningDatasetConfig` uses `seq_length`. + +## LLM Pretraining Data + +LLM pretraining uses Megatron binary indexed datasets. Each dataset is represented by a prefix with matching `.bin` and `.idx` files: + +```text +/data/dclm/preprocessed_text_document.bin +/data/dclm/preprocessed_text_document.idx +``` + +Pass the prefix without the `.bin` or `.idx` suffix: + +```python +from megatron.bridge.training.config import GPTDatasetConfig + +dataset = GPTDatasetConfig( + seq_length=8192, + data_path="/data/dclm/preprocessed_text_document", + split="9999,8,2", + random_seed=1234, + reset_attention_mask=False, + reset_position_ids=False, + eod_mask_loss=False, +) +``` + +The CLI-friendly `data_path` field is converted to Megatron Core's `blend` field during config finalization. For weighted multi-dataset training, use either a flattened `data_path` list with weights and prefixes or set `blend`/`blend_per_split` directly. + +```bash +uv run python -m torch.distributed.run --nproc_per_node=8 scripts/training/run_recipe.py \ + --recipe llama32_1b_pretrain_config \ + --dataset llm-pretrain \ + dataset.data_path=/data/dclm/preprocessed_text_document \ + dataset.seq_length=8192 +``` + +To create Megatron binary data from JSONL text, use the Megatron-LM `tools/preprocess_data.py` workflow. The DCLM tutorial shows a complete download, merge, shuffle, and preprocessing flow: [DCLM Data Preprocessing Tutorial](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/tutorials/data/dclm/README.md). + +## Local JSONL SFT and PEFT Data + +Text SFT and PEFT use a directory containing split files named `training.jsonl`, `validation.jsonl`, and optionally `test.jsonl`: + +```text +/data/sft_jsonl/ + training.jsonl + validation.jsonl + test.jsonl +``` + +The default text SFT dataset expects each JSONL record to contain prompt and answer fields compatible with the configured `prompt_template`. The common input/output format is: + +```json +{"input": "Question: What is Megatron Bridge?", "output": "A PyTorch-native bridge for Megatron-Core workflows."} +``` + +Configure local JSONL data with `FinetuningDatasetConfig.dataset_root`: + +```python +from megatron.bridge.training.config import FinetuningDatasetConfig + +dataset = FinetuningDatasetConfig( + dataset_root="/data/sft_jsonl", + seq_length=4096, +) +``` + +Launch the generic recipe runner with the preloaded local JSONL dataset type: + +```bash +uv run python -m torch.distributed.run --nproc_per_node=8 scripts/training/run_recipe.py \ + --recipe llama32_1b_sft_config \ + --dataset llm-finetune-preloaded \ + dataset.dataset_root=/data/sft_jsonl \ + dataset.seq_length=4096 \ + checkpoint.pretrained_checkpoint=/checkpoints/base_model +``` + +For PEFT, use the PEFT recipe or set `cfg.peft`; the data layout stays the same. `checkpoint.pretrained_checkpoint` is required for the frozen base model, and `checkpoint.load` is used only when resuming adapter checkpoints. + +## Hugging Face Datasets for SFT and PEFT + +`HFDatasetConfig` downloads or reads a Hugging Face dataset, applies a processing function to each example, writes Bridge-compatible JSONL split files, and then builds the same fine-tuning dataset used by local JSONL. + +```python +from megatron.bridge.data.builders.hf_dataset import HFDatasetConfig +from megatron.bridge.data.hf_processors.squad import process_squad_example + +dataset = HFDatasetConfig( + dataset_name="rajpurkar/squad", + process_example_fn=process_squad_example, + dataset_root="/data/processed/squad", + seq_length=512, + val_proportion=0.1, + do_test=False, +) +``` + +If `dataset_root` is omitted, the processed JSONL is cached under the NeMo datasets cache for the dataset name. Set `rewrite=False` when you want later runs to reuse already processed files. + +The generic launcher provides preset Hugging Face text datasets through `--dataset llm-finetune`: + +```bash +uv run python -m torch.distributed.run --nproc_per_node=8 scripts/training/run_recipe.py \ + --recipe llama32_1b_peft_config \ + --dataset llm-finetune \ + dataset.dataset_name=gsm8k \ + checkpoint.pretrained_checkpoint=/checkpoints/base_model +``` + +## VLM Fine-Tuning Data + +VLM recipes usually use a dataset provider instead of `FinetuningDatasetConfig`. The provider owns both the storage format and the processor needed to turn image, video, audio, and text records into batches. + +For Energon/WebDataset data, create tar shards plus `.nv-meta` metadata and pass the dataset root to the recipe provider: + +```bash +uv run python -m torch.distributed.run --nproc_per_node=8 scripts/training/run_recipe.py \ + --recipe qwen3_vl_8b_peft_energon_config \ + --dataset vlm-energon \ + --step_func qwen3_vl_step \ + dataset.path=/data/vlm_energon \ + checkpoint.pretrained_checkpoint=/checkpoints/qwen3_vl_base +``` + +For preloaded VLM JSON or JSONL, use records with `messages` or `conversations` plus media paths. Relative image and video paths are resolved against `dataset.image_folder` by `PreloadedVLMConversationProvider`: + +```json +{"messages": [{"role": "user", "content": "Describe the image."}, {"role": "assistant", "content": "A receipt."}], "images": ["receipt_0001.jpg"]} +``` + +```bash +uv run python -m torch.distributed.run --nproc_per_node=8 scripts/training/run_recipe.py \ + --recipe qwen3_vl_8b_peft_config \ + --dataset vlm-preloaded \ + --step_func qwen3_vl_step \ + dataset.train_data_path=/data/vlm/train.jsonl \ + dataset.valid_data_path=/data/vlm/validation.jsonl \ + dataset.image_folder=/data/vlm/images \ + dataset.hf_processor_path=Qwen/Qwen3-VL-8B-Instruct \ + checkpoint.pretrained_checkpoint=/checkpoints/qwen3_vl_base +``` + +For a complete WebDataset/Energon preparation example, see [VALOR32K-AVQA Dataset Preparation Guide](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/tutorials/data/valor32k-avqa/data-preparation.md). + +## Checkpoint Conversion Reminder + +Data preparation and checkpoint preparation are separate. From-scratch pretraining does not require a checkpoint. SFT and PEFT require base model weights through `checkpoint.pretrained_checkpoint` unless you are resuming from a complete native Megatron checkpoint with `checkpoint.load`. + +`checkpoint.pretrained_checkpoint` may point to a native Megatron checkpoint directory, a specific native `iter_N` directory, or a local Hugging Face full-model directory. For production and multi-node jobs, converting Hugging Face checkpoints to native Megatron format first is usually more repeatable. diff --git a/docs/training/entry-points.md b/docs/training/entry-points.mdx similarity index 90% rename from docs/training/entry-points.md rename to docs/training/entry-points.mdx index cfce66be92..a9feb57e29 100644 --- a/docs/training/entry-points.md +++ b/docs/training/entry-points.mdx @@ -1,3 +1,8 @@ +--- +title: "Training Entry Points" +description: "" +--- + # Training Entry Points Megatron Bridge provides unified training entry points for pretraining, Supervised Fine-Tuning (SFT), and Parameter-Efficient Fine-Tuning (PEFT). All training modes share the same underlying training loop architecture, differing primarily in their data handling and model configuration. @@ -24,7 +29,7 @@ For multi-node jobs and repeatable production runs, converting a Hugging Face mo ## Main Entry Points -The {py:func}`bridge.training.pretrain.pretrain` and {py:func}`bridge.training.finetune.finetune` functions are the primary entry points for pretraining modelsโ€”either from scratch or through fine-tuning. Each function accepts a {py:class}`bridge.training.config.ConfigContainer` along with a `forward_step_func` that defines how the training loop should be run. +The `bridge.training.pretrain.pretrain` and `bridge.training.finetune.finetune` functions are the primary entry points for pretraining modelsโ€”either from scratch or through fine-tuning. Each function accepts a `bridge.training.config.ConfigContainer` along with a `forward_step_func` that defines how the training loop should be run. ## Forward Step Function @@ -63,7 +68,7 @@ The forward step function has three main responsibilities: ### State Access -Megatron Bridge automatically provides the {py:class}`bridge.training.state.GlobalState` object containing: +Megatron Bridge automatically provides the `bridge.training.state.GlobalState` object containing: - **Configuration**: Complete training configuration (`global_state.cfg`). - **Timers**: Performance monitoring utilities (`global_state.timers`). - **Training Progress**: Current step, consumed samples (`global_state.train_state`). @@ -71,7 +76,7 @@ Megatron Bridge automatically provides the {py:class}`bridge.training.state.Glob All configuration and state information are accessible through the injected `state` object. -For complete implementation examples, see {py:func}`bridge.training.gpt_step.forward_step`. +For complete implementation examples, see `bridge.training.gpt_step.forward_step`. ## Loss Calculation and Reduction @@ -102,7 +107,7 @@ The training loop automatically handles: - **Pipeline Coordination**: Only the last pipeline stage computes and reduces losses. - **Logging Integration**: Automatically logs loss components to TensorBoard/WandB. -For implementation details, see {py:func}`bridge.training.train.train_step` and {py:func}`bridge.training.losses.masked_token_loss`, as an example. +For implementation details, see `bridge.training.train.train_step` and `bridge.training.losses.masked_token_loss`, as an example. ## Customization diff --git a/docs/training/hierarchical-context-parallel.md b/docs/training/hierarchical-context-parallel.mdx similarity index 85% rename from docs/training/hierarchical-context-parallel.md rename to docs/training/hierarchical-context-parallel.mdx index b5516d425c..7f98609d92 100644 --- a/docs/training/hierarchical-context-parallel.md +++ b/docs/training/hierarchical-context-parallel.mdx @@ -1,3 +1,8 @@ +--- +title: "Hierarchical Context Parallel" +description: "" +--- + # Hierarchical Context Parallel This page covers hierarchical context parallelism in Megatron Bridge, @@ -5,7 +10,7 @@ especially the `a2a+p2p` transport path and `hierarchical_context_parallel_sizes`. For operational setup, code anchors, and verification commands, see -[the hierarchical CP skill](../skills/nemo-mbridge-perf-hierarchical-context-parallel/SKILL.md). +[the hierarchical CP skill](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-hierarchical-context-parallel/SKILL.md). ## What It Is @@ -102,7 +107,7 @@ all Bridge initialization modes. ## Related Docs -- [docs/performance-guide.md](../performance-guide.md) -- [docs/training/communication-overlap.md](communication-overlap.md) -- [Hierarchical CP skill](../skills/nemo-mbridge-perf-hierarchical-context-parallel/SKILL.md) -- [Hierarchical CP skill card](../skills/nemo-mbridge-perf-hierarchical-context-parallel/card.yaml) +- [docs/training/performance-tuning.mdx](/training/performance-tuning) +- [docs/training/communication-overlap.mdx](/training/communication-overlap) +- [Hierarchical CP skill](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-hierarchical-context-parallel/SKILL.md) +- [Hierarchical CP skill card](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-hierarchical-context-parallel/card.yaml) diff --git a/docs/training/index.mdx b/docs/training/index.mdx new file mode 100644 index 0000000000..9e51dfcde0 --- /dev/null +++ b/docs/training/index.mdx @@ -0,0 +1,129 @@ +--- +title: "Training and Customization" +description: "" +--- + +# Training and Customization + +This directory contains comprehensive documentation for training and customizing models with Megatron Bridge. Learn how to configure training, optimize performance, and customize training workflows. + +## Quick Navigation + +### I want to + +**๐Ÿš€ Get started with training** +-> Start with [Configuration Container Overview](/training/config-container-overview) to understand the training setup + +**โš™๏ธ Configure training parameters** +-> See [Training Loop Settings](/training/training-loop-settings) and [Optimizer & Scheduler](/training/optimizer-scheduler) + +**๐Ÿ“Š Monitor and profile training** +-> Check [Logging](/training/logging) and [Profiling](/training/profiling) guides + +**๐Ÿ’พ Manage checkpoints** +-> Read [Checkpointing](/training/checkpointing) for saving and resuming training + +**โšก Optimize performance** +-> Explore [Performance Guide](/training/performance-tuning) and [Performance Summary](/reference/performance/summary) + +**๐Ÿ”ง Customize training** +-> See [PEFT](/training/peft), [Distillation](/model-optimization/distillation), [Entry Points](/training/entry-points), and [Callbacks](/training/callbacks) + +## Core Training Documentation + +### Configuration and Setup + +| Document | Purpose | When to Read | +|----------|---------|--------------| +| **[Configuration Container Overview](/training/config-container-overview)** | Central configuration object for all training settings | First time setting up training | +| **[Entry Points](/training/entry-points)** | Training entry points and execution flow | Understanding how training starts | +| **[Training Loop Settings](/training/training-loop-settings)** | Training loop parameters and configuration | Configuring batch sizes, iterations, validation | + +### Optimization and Performance + +| Document | Purpose | When to Read | +|----------|---------|--------------| +| **[Optimizer & Scheduler](/training/optimizer-scheduler)** | Optimizer and learning rate scheduler configuration | Setting up optimization | +| **[Mixed Precision](/training/mixed-precision)** | Mixed precision training for memory efficiency | Reducing memory usage | +| **[Communication Overlap](/training/communication-overlap)** | Overlapping communication with computation | Optimizing distributed training | +| **[Hierarchical Context Parallel](/training/hierarchical-context-parallel)** | Hierarchical `a2a+p2p` context parallel guidance | Advanced long-sequence scaling | +| **[Attention Optimizations](/training/attention-optimizations)** | Optimizing attention mechanisms | Improving training speed | +| **[Activation Recomputation](/training/activation-recomputation)** | Gradient checkpointing strategies | Reducing memory footprint | +| **[CPU Offloading](/training/cpu-offloading)** | Offloading to CPU for memory management | Working with limited GPU memory | + +### Monitoring and Debugging + +| Document | Purpose | When to Read | +|----------|---------|--------------| +| **[Logging](/training/logging)** | Logging configuration and TensorBoard/WandB integration | Monitoring training progress | +| **[Profiling](/training/profiling)** | Performance profiling and analysis | Identifying bottlenecks | +| **[Theoretical Memory Estimator](/training/memory-estimator)** | Formula-based per-GPU memory planning | Sizing dense and MoE training configs | +| **[Resiliency](/training/resiliency)** | Handling failures and recovery | Building robust training pipelines | + +### Advanced Features + +| Document | Purpose | When to Read | +|----------|---------|--------------| +| **[PEFT](/training/peft)** | Parameter-Efficient Fine-Tuning (LoRA, etc.) | Fine-tuning with limited resources | +| **[Packed Sequences](/training/packed-sequences)** | Sequence packing for efficiency | Optimizing data loading | +| **[Megatron FSDP](/training/megatron-fsdp)** | Stable overview of Megatron FSDP | Choosing an FSDP path | +| **[Distillation](/model-optimization/distillation)** | Knowledge distillation techniques | Transferring knowledge between models | +| **[Checkpointing](/training/checkpointing)** | Checkpoint saving, loading, and resuming | Managing training state | +| **[Callbacks](/training/callbacks)** | Inject custom logic into training loop | Custom logging, metrics, third-party integrations | + +## Training Workflow + +A typical training workflow involves: + +1. **Configure Training** - Set up `ConfigContainer` with model, data, and training parameters +2. **Prepare Data** - Configure dataset loading and preprocessing +3. **Set Optimization** - Configure optimizer, scheduler, and mixed precision +4. **Enable Monitoring** - Set up logging and profiling +5. **Configure Checkpointing** - Set up checkpoint saving and resuming +6. **Launch Training** - Start training with configured entry points +7. **Monitor Progress** - Track metrics via logging and profiling +8. **Resume if Needed** - Use checkpointing to resume from saved state + +## Related Documentation + +- **[Main Documentation Index](/)** - Return to main documentation +- **[Performance Guide](/training/performance-tuning)** - Comprehensive performance optimization guide +- **[Performance Summary](/reference/performance/summary)** - Quick performance reference +- **[Recipe Usage](/training/recipe-usage)** - Using training recipes +- **[Parallelisms](/training/parallelisms)** - Understanding distributed training strategies +- **[Bridge Guide](/get-started/quickstart/hugging-face-conversion)** - Working with Hugging Face models + +## Common Training Scenarios + +### ๐Ÿ†• First-Time Training Setup + +1. [Configuration Container Overview](/training/config-container-overview) - Understand the configuration system +2. [Entry Points](/training/entry-points) - Learn how to start training +3. [Training Loop Settings](/training/training-loop-settings) - Configure basic training parameters +4. [Logging](/training/logging) - Set up monitoring + +### โšก Performance Optimization + +1. [Performance Guide](/training/performance-tuning) - Comprehensive optimization strategies +2. [Mixed Precision](/training/mixed-precision) - Enable mixed precision training +3. [Communication Overlap](/training/communication-overlap) - Optimize distributed training +4. [Activation Recomputation](/training/activation-recomputation) - Reduce memory usage +5. [Profiling](/training/profiling) - Identify bottlenecks + +### ๐Ÿ’พ Production Training + +1. [Checkpointing](/training/checkpointing) - Reliable checkpoint management +2. [Resiliency](/training/resiliency) - Handle failures gracefully +3. [Logging](/training/logging) - Comprehensive monitoring +4. [Profiling](/training/profiling) - Performance analysis + +### ๐Ÿ”ง Customization + +1. [PEFT](/training/peft) - Parameter-efficient fine-tuning +2. [Distillation](/model-optimization/distillation) - Knowledge distillation +3. [Entry Points](/training/entry-points) - Custom training workflows +4. [Callbacks](/training/callbacks) - Inject custom logic (third-party integrations) + +--- + +**Ready to start training?** Begin with [Configuration Container Overview](/training/config-container-overview) or return to the [main documentation](/resources/documentation-map). diff --git a/docs/training/logging.md b/docs/training/logging.mdx similarity index 93% rename from docs/training/logging.md rename to docs/training/logging.mdx index d07392c7ab..6f36d754c3 100644 --- a/docs/training/logging.md +++ b/docs/training/logging.mdx @@ -1,10 +1,15 @@ +--- +title: "Logging and Monitoring" +description: "" +--- + # Logging and Monitoring This guide describes how to configure logging in Megatron Bridge. It introduces the high-level `LoggerConfig`, explains experiment logging to TensorBoard and Weights & Biases (W&B), and documents console logging behavior. ## LoggerConfig Overview -{py:class}`~bridge.training.config.LoggerConfig` is the dataclass that encapsulates loggingโ€‘related settings for training. It resides inside the overall {py:class}`bridge.training.config.ConfigContainer`, which represents the complete configuration for a training run. +`bridge.training.config.LoggerConfig` is the dataclass that encapsulates loggingโ€‘related settings for training. It resides inside the overall `bridge.training.config.ConfigContainer`, which represents the complete configuration for a training run. ### Timer Configuration Options @@ -86,9 +91,9 @@ TensorBoard captures a range of training and system metrics, including: ) ``` - ```{note} - The writer is created lazily on the last rank when `tensorboard_dir` is set. - ``` + + The writer is created lazily on the last rank when `tensorboard_dir` is set. + #### Set the Output Directory @@ -136,13 +141,13 @@ In addition, the full run configuration is synced at initialization, allowing fo ) ``` -```{note} + W&B is initialized lazily on the last rank when `wandb_project` is set and `wandb_exp_name` is non-empty. -``` + #### W&B Configuration with NeMo Run Launching -For users launching training scripts with NeMo Run, W&B can be optionally configured using the {py:class}`bridge.recipes.run_plugins.WandbPlugin`. +For users launching training scripts with NeMo Run, W&B can be optionally configured using the `bridge.recipes.run_plugins.WandbPlugin`. The plugin automatically forwards the `WANDB_API_KEY` and by default injects CLI overrides for the following logger parameters: @@ -236,13 +241,13 @@ When enabled, Comet ML receives: ) ``` -```{note} + Comet ML is initialized lazily on the last rank when `comet_project` is set and `comet_experiment_name` is non-empty. -``` + #### Comet ML Configuration with NeMo Run Launching -For users launching training scripts with NeMo Run, Comet ML can be optionally configured using the {py:class}`bridge.recipes.run_plugins.CometPlugin`. +For users launching training scripts with NeMo Run, Comet ML can be optionally configured using the `bridge.recipes.run_plugins.CometPlugin`. The plugin automatically forwards the `COMET_API_KEY` and by default injects CLI overrides for the following logger parameters: @@ -270,18 +275,18 @@ This provides a lightweight, text-based audit trail of training progress, useful ## Tensor Inspection -Megatron Bridge integrates with TransformerEngine's tensor inspection features via NVIDIA DLFW Inspect. This integration, controlled by {py:class}`~bridge.training.config.TensorInspectConfig`, enables advanced debugging and analysis of tensor statistics during training. When enabled, the framework handles initialization, step tracking, and cleanup automatically. +Megatron Bridge integrates with TransformerEngine's tensor inspection features via NVIDIA DLFW Inspect. This integration, controlled by `bridge.training.config.TensorInspectConfig`, enables advanced debugging and analysis of tensor statistics during training. When enabled, the framework handles initialization, step tracking, and cleanup automatically. -```{note} + **Current limitations:** Tensor inspection is currently supported only for linear modules in TransformerEngine (e.g., `fc1`, `fc2`, `layernorm_linear`). Operations like attention are not supported. -``` + -```{note} + This section covers Megatron Bridge configuration. For comprehensive documentation on features, configuration syntax, and advanced usage, see: - [TransformerEngine Debug Documentation](https://github.com/NVIDIA/TransformerEngine/tree/af2a0c16ec11363c0af84690cd877a59f898820e/docs/debug) - [NVIDIA DLFW Inspect Documentation](https://github.com/NVIDIA/nvidia-dlfw-inspect/tree/4118044cc84f0183714a2ab1bc215fa49f9aaa82/docs) -``` + ### Installation @@ -305,7 +310,7 @@ See [TransformerEngine debug features](https://github.com/NVIDIA/TransformerEngi ### Configuration -Configure tensor inspection using {py:class}`~bridge.training.config.TensorInspectConfig` with either a YAML file or inline dictionary. +Configure tensor inspection using `bridge.training.config.TensorInspectConfig` with either a YAML file or inline dictionary. #### YAML Configuration @@ -436,4 +441,3 @@ Straggler timing reports follow the same `log_interval` cadence, helping identif To reduce performance impact, set `timing_log_level` to `0`. Increase to `1` or `2` only when more detailed timing metrics are required, as higher levels introduce additional logging overhead. - diff --git a/docs/training/megatron-fsdp.md b/docs/training/megatron-fsdp.mdx similarity index 86% rename from docs/training/megatron-fsdp.md rename to docs/training/megatron-fsdp.mdx index 17f58fafe5..1876cbde59 100644 --- a/docs/training/megatron-fsdp.md +++ b/docs/training/megatron-fsdp.mdx @@ -1,3 +1,8 @@ +--- +title: "Megatron FSDP" +description: "" +--- + # Megatron FSDP Megatron FSDP is the practical fully sharded data parallel path in Megatron @@ -7,7 +12,7 @@ plain Distributed Data Parallel (DDP) or the distributed optimizer path. This page is the stable overview for what Megatron FSDP is, when to use it, and what constraints matter. For operational enablement, code anchors, and -verification commands, see [skills/nemo-mbridge-perf-megatron-fsdp/SKILL.md](../skills/nemo-mbridge-perf-megatron-fsdp/SKILL.md). +verification commands, see [skills/nemo-mbridge-perf-megatron-fsdp/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-megatron-fsdp/SKILL.md). ## What It Is @@ -109,8 +114,8 @@ The stable recommendation today is: ## Related Docs -- [docs/training/checkpointing.md](checkpointing.md) -- [docs/training/cpu-offloading.md](cpu-offloading.md) -- [docs/performance-guide.md](../performance-guide.md) -- [skills/nemo-mbridge-perf-megatron-fsdp/SKILL.md](../skills/nemo-mbridge-perf-megatron-fsdp/SKILL.md) -- [skills/nemo-mbridge-perf-megatron-fsdp/card.yaml](../skills/nemo-mbridge-perf-megatron-fsdp/card.yaml) +- [docs/training/checkpointing.mdx](/training/checkpointing) +- [docs/training/cpu-offloading.mdx](/training/cpu-offloading) +- [docs/training/performance-tuning.mdx](/training/performance-tuning) +- [skills/nemo-mbridge-perf-megatron-fsdp/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-megatron-fsdp/SKILL.md) +- [skills/nemo-mbridge-perf-megatron-fsdp/card.yaml](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-megatron-fsdp/card.yaml) diff --git a/docs/training/memory-estimator.md b/docs/training/memory-estimator.mdx similarity index 92% rename from docs/training/memory-estimator.md rename to docs/training/memory-estimator.mdx index d7220b9331..bc060c9c0d 100644 --- a/docs/training/memory-estimator.md +++ b/docs/training/memory-estimator.mdx @@ -1,3 +1,8 @@ +--- +title: "Theoretical Memory Estimator" +description: "" +--- + # Theoretical Memory Estimator Megatron Bridge includes a formula-based training memory estimator for GPT-like @@ -68,6 +73,6 @@ normalization, and embedding parameters. This keeps expert parallelism effects visible in the returned component breakdown. For memory tuning decisions, use this estimator as a planning signal together -with the runtime guidance in [CPU Offloading](cpu-offloading.md), -[Activation Recomputation](activation-recomputation.md), and -[Megatron FSDP](megatron-fsdp.md). +with the runtime guidance in [CPU Offloading](/training/cpu-offloading), +[Activation Recomputation](/training/activation-recomputation), and +[Megatron FSDP](/training/megatron-fsdp). diff --git a/docs/training/mixed-precision.md b/docs/training/mixed-precision.mdx similarity index 90% rename from docs/training/mixed-precision.md rename to docs/training/mixed-precision.mdx index 7558880c63..3a51b691f3 100644 --- a/docs/training/mixed-precision.md +++ b/docs/training/mixed-precision.mdx @@ -1,12 +1,17 @@ +--- +title: "Mixed Precision Training" +description: "" +--- + # Mixed Precision Training -Mixed precision training significantly enhances computational efficiency by conducting operations in low-precision format, while selectively maintaining minimal data in single-precision to preserve critical information throughout key areas of the network. Megatron Bridge supports FP16, BF16, and FP8 via Transformer Engine (TE) across most models through the {py:class}`bridge.training.mixed_precision.MixedPrecisionConfig` configuration. +Mixed precision training significantly enhances computational efficiency by conducting operations in low-precision format, while selectively maintaining minimal data in single-precision to preserve critical information throughout key areas of the network. Megatron Bridge supports FP16, BF16, and FP8 via Transformer Engine (TE) across most models through the `bridge.training.mixed_precision.MixedPrecisionConfig` configuration. ## Configuration Overview -Mixed precision is configured in Megatron Bridge through the `mixed_precision` field in {py:class}`bridge.training.config.ConfigContainer`, which accepts either: +Mixed precision is configured in Megatron Bridge through the `mixed_precision` field in `bridge.training.config.ConfigContainer`, which accepts either: - A string name referencing a predefined recipe (e.g., `"bf16_mixed"`) -- A {py:class}`bridge.training.mixed_precision.MixedPrecisionConfig` object for custom configurations +- A `bridge.training.mixed_precision.MixedPrecisionConfig` object for custom configurations The mixed precision configuration automatically updates the model, optimizer, and distributed data parallel settings with the appropriate precision parameters. @@ -36,7 +41,7 @@ config = ConfigContainer( ### Custom Mixed Precision Configuration -For more control, create a custom {py:class}`bridge.training.mixed_precision.MixedPrecisionConfig`: +For more control, create a custom `bridge.training.mixed_precision.MixedPrecisionConfig`: ```python from megatron.bridge.training.config import ConfigContainer @@ -64,7 +69,7 @@ NVIDIA H100 GPU introduced support for a new datatype, FP8 (8-bit floating point ### FP8 Configuration Parameters -The {py:class}`bridge.training.mixed_precision.MixedPrecisionConfig` provides several FP8-specific parameters: +The `bridge.training.mixed_precision.MixedPrecisionConfig` provides several FP8-specific parameters: | Parameter | Type | Default | Description | |-----------|------|---------|-------------| @@ -141,7 +146,7 @@ NVFP4 is related to FP8 in that both use Transformer Engine low-precision kernel ## Available Mixed Precision Recipes -Megatron Bridge provides numerous predefined mixed precision recipes for different use cases. You can use the {py:func}`~megatron.bridge.training.mixed_precision.get_mixed_precision_config` utility function to convert from a string shortname to a class instance. For the complete list of available recipes and their specific configurations, see the {py:mod}`megatron.bridge.training.mixed_precision` module. +Megatron Bridge provides numerous predefined mixed precision recipes for different use cases. You can use the `megatron.bridge.training.mixed_precision.get_mixed_precision_config` utility function to convert from a string shortname to a class instance. For the complete list of available recipes and their specific configurations, see the `megatron.bridge.training.mixed_precision` module. ### Custom FP8 Configuration @@ -173,7 +178,7 @@ config = ConfigContainer( ### Registering Custom Mixed Precision Recipes -You can also register your own custom mixed precision configurations to work with the shortname system. Use the {py:func}`~megatron.bridge.training.mixed_precision.register` decorator on a function that returns a `MixedPrecisionConfig` object: +You can also register your own custom mixed precision configurations to work with the shortname system. Use the `megatron.bridge.training.mixed_precision.register` decorator on a function that returns a `MixedPrecisionConfig` object: ```python from megatron.bridge.training.mixed_precision import register, MixedPrecisionConfig diff --git a/docs/training/moe-optimization.md b/docs/training/moe-optimization.mdx similarity index 93% rename from docs/training/moe-optimization.md rename to docs/training/moe-optimization.mdx index 31cef195df..efd8cf8139 100644 --- a/docs/training/moe-optimization.md +++ b/docs/training/moe-optimization.mdx @@ -1,3 +1,8 @@ +--- +title: "MoE Training Optimization" +description: "" +--- + # MoE Training Optimization This page covers the optimization framework, techniques, and best practices for @@ -6,12 +11,12 @@ training Mixture-of-Experts models with Megatron-Core. It is based on For tuning knobs, hardware-specific configs, and benchmark-oriented guidance, see: -- [skills/nemo-mbridge-perf-moe-optimization-workflow/SKILL.md](../skills/nemo-mbridge-perf-moe-optimization-workflow/SKILL.md) -- [skills/nemo-mbridge-perf-moe-long-context/SKILL.md](../skills/nemo-mbridge-perf-moe-long-context/SKILL.md) -- [skills/nemo-mbridge-perf-moe-vlm-training/SKILL.md](../skills/nemo-mbridge-perf-moe-vlm-training/SKILL.md) -- [skills/nemo-mbridge-perf-moe-dispatcher-selection/SKILL.md](../skills/nemo-mbridge-perf-moe-dispatcher-selection/SKILL.md) -- [skills/nemo-mbridge-perf-moe-hardware-configs/SKILL.md](../skills/nemo-mbridge-perf-moe-hardware-configs/SKILL.md) -- [skills/nemo-mbridge-perf-moe-comm-overlap/SKILL.md](../skills/nemo-mbridge-perf-moe-comm-overlap/SKILL.md) +- [skills/nemo-mbridge-perf-moe-optimization-workflow/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-moe-optimization-workflow/SKILL.md) +- [skills/nemo-mbridge-perf-moe-long-context/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-moe-long-context/SKILL.md) +- [skills/nemo-mbridge-perf-moe-vlm-training/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-moe-vlm-training/SKILL.md) +- [skills/nemo-mbridge-perf-moe-dispatcher-selection/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-moe-dispatcher-selection/SKILL.md) +- [skills/nemo-mbridge-perf-moe-hardware-configs/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-moe-hardware-configs/SKILL.md) +- [skills/nemo-mbridge-perf-moe-comm-overlap/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-moe-comm-overlap/SKILL.md) ## The Three Walls @@ -230,7 +235,7 @@ PP layout string controls per-stage layer distribution: ## EP Communication Overlap For exact overlap constraints and verification guidance, see -[docs/training/communication-overlap.md](communication-overlap.md) and the +[docs/training/communication-overlap.mdx](/training/communication-overlap) and the related MoE overlap skills. Two overlap patterns for 1F1B: diff --git a/docs/training/multi-token-prediction.md b/docs/training/multi-token-prediction.mdx similarity index 92% rename from docs/training/multi-token-prediction.md rename to docs/training/multi-token-prediction.mdx index 200865342c..f3a57dbbff 100644 --- a/docs/training/multi-token-prediction.md +++ b/docs/training/multi-token-prediction.mdx @@ -1,3 +1,8 @@ +--- +title: "Multi-Token Prediction (MTP)" +description: "" +--- + # Multi-Token Prediction (MTP) ## Overview @@ -57,7 +62,7 @@ where: - Default `0.1` works well for most models - Increase to `0.15-0.2` if MTP losses aren't decreasing - Decrease to `0.05-0.08` if main loss is being overshadowed -- Scale factor should be proportional to `mtp_num_layers` (more layers โ†’ lower factor) +- Scale factor should be proportional to `mtp_num_layers` (more layers -> lower factor) ## Basic Usage: Training from Scratch @@ -194,7 +199,7 @@ The following features are not yet supported with MTP: | Feature | Status | Workaround | |---------|--------|------------| -| **HuggingFace โ†” Megatron Checkpoint Conversion** | โš ๏ธ Model-specific | Conversion support varies by model; check model-specific documentation | +| **Hugging Face to and from Megatron Checkpoint Conversion** | โš ๏ธ Model-specific | Conversion support varies by model; check model-specific documentation | | **Sequence Packing (Fine-Tuning)** | โŒ Not supported | For pre-training, no issues. For fine-tuning, set `packed_sequence_specs=None` | | **Cross-Attention** | โŒ Not supported | MTP only works with decoder-only models (GPT, Llama, etc.) | | **Learned Absolute Position Embeddings** | โŒ Not supported | Use RoPE (rotary position embeddings) or no position embeddings | @@ -204,7 +209,7 @@ The following features are not yet supported with MTP: **Checkpoint Conversion:** -HuggingFace โ†” Megatron checkpoint conversion with MTP is model-specific. Some models have conversion support planned, while others may not support MTP parameter mapping. Check the documentation for your specific model. +Hugging Face to and from Megatron checkpoint conversion with MTP is model-specific. Some models have conversion support planned, while others may not support MTP parameter mapping. Check the documentation for your specific model. **Sequence Packing:** @@ -236,15 +241,15 @@ This is normal. Only the rank selected by the pipeline layout builds MTP layers. ### Code Examples -- **DeepSeek-V3 Recipe**: [`src/megatron/bridge/recipes/deepseek/deepseek_v3.py`](../../src/megatron/bridge/recipes/deepseek/deepseek_v3.py) +- **DeepSeek-V3 Recipe**: [`src/megatron/bridge/recipes/deepseek/deepseek_v3.py`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/recipes/deepseek/deepseek_v3.py) - Example of MTP with large-scale MoE model - Predefined pipeline layouts for PP + MTP -- **Qwen3-Next Recipe**: [`src/megatron/bridge/recipes/qwen/qwen3_next.py`](../../src/megatron/bridge/recipes/qwen/qwen3_next.py) +- **Qwen3-Next Recipe**: [`src/megatron/bridge/recipes/qwen/qwen3_next.py`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/recipes/qwen/qwen3_next.py) - Clean example of MTP configuration for dense models - Good starting point for custom recipes -- **MTP Core Implementation**: [`3rdparty/Megatron-LM/megatron/core/transformer/multi_token_prediction.py`](../../3rdparty/Megatron-LM/megatron/core/transformer/multi_token_prediction.py) +- **MTP Core Implementation**: [`3rdparty/Megatron-LM/megatron/core/transformer/multi_token_prediction.py`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/3rdparty/Megatron-LM/megatron/core/transformer/multi_token_prediction.py) - Low-level MTP layer implementation - Loss computation and logging helpers @@ -253,7 +258,7 @@ This is normal. Only the rank selected by the pipeline layout builds MTP layers. - **Megatron Core MTP Guide**: [`3rdparty/Megatron-LM/docs/user-guide/features/multi_token_prediction.md`](https://github.com/NVIDIA/Megatron-LM/blob/main/docs/user-guide/features/multi_token_prediction.md) - Implementation notes and design decisions -- **Pipeline Parallelism Guide**: [`docs/parallelisms.md`](../parallelisms.md) +- **Pipeline Parallelism Guide**: [`docs/training/parallelisms.mdx`](/training/parallelisms) - Understanding pipeline parallelism layouts - Best practices for PP configuration diff --git a/docs/training/optimizer-scheduler.md b/docs/training/optimizer-scheduler.mdx similarity index 98% rename from docs/training/optimizer-scheduler.md rename to docs/training/optimizer-scheduler.mdx index 22b5f144d7..98dd2643cc 100644 --- a/docs/training/optimizer-scheduler.md +++ b/docs/training/optimizer-scheduler.mdx @@ -1,3 +1,8 @@ +--- +title: "Optimizer and Scheduler Configuration" +description: "" +--- + # Optimizer and Scheduler Configuration The optimizer and scheduler configurations control optimization algorithms, learning rate schedules, and weight decay strategies. @@ -113,4 +118,4 @@ These scheduling options control how the weight decay coefficient changes over t |---------------|-----------------------------------------------------------------------------| | **Constant** | Fixed weight decay throughout training. | | **Linear** | Linear progression from start to end weight decay. | -| **Cosine** | Cosine progression from start to end weight decay. | \ No newline at end of file +| **Cosine** | Cosine progression from start to end weight decay. | diff --git a/docs/training/packed-sequences.md b/docs/training/packed-sequences.mdx similarity index 80% rename from docs/training/packed-sequences.md rename to docs/training/packed-sequences.mdx index 5eb53db3a8..523bceecd6 100644 --- a/docs/training/packed-sequences.md +++ b/docs/training/packed-sequences.mdx @@ -1,3 +1,8 @@ +--- +title: "Packed Sequences" +description: "" +--- + # Packed Sequences Packed sequences are a fine-tuning technique that reduces padding waste by @@ -7,7 +12,7 @@ fine-tuning and PEFT optimization rather than a general pretraining feature. This page is the stable overview for what packed sequences are, when to use them, and which constraints are durable. For operational setup, code anchors, -and verification commands, see [skills/nemo-mbridge-perf-sequence-packing/SKILL.md](../skills/nemo-mbridge-perf-sequence-packing/SKILL.md). +and verification commands, see [skills/nemo-mbridge-perf-sequence-packing/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-sequence-packing/SKILL.md). ## What It Is @@ -72,8 +77,8 @@ problems: For long-sequence training guidance, see: -- `docs/performance-guide.md` -- `docs/training/hierarchical-context-parallel.md` +- `docs/training/performance-tuning.mdx` +- `docs/training/hierarchical-context-parallel.mdx` ## Practical Caveats @@ -87,8 +92,8 @@ The most stable caveats to remember are: ## Related Docs -- [docs/training/multi-token-prediction.md](multi-token-prediction.md) -- [docs/performance-guide.md](../performance-guide.md) -- [docs/training/hierarchical-context-parallel.md](hierarchical-context-parallel.md) -- [skills/nemo-mbridge-perf-sequence-packing/SKILL.md](../skills/nemo-mbridge-perf-sequence-packing/SKILL.md) -- [skills/nemo-mbridge-perf-sequence-packing/card.yaml](../skills/nemo-mbridge-perf-sequence-packing/card.yaml) +- [docs/training/multi-token-prediction.mdx](/training/multi-token-prediction) +- [docs/training/performance-tuning.mdx](/training/performance-tuning) +- [docs/training/hierarchical-context-parallel.mdx](/training/hierarchical-context-parallel) +- [skills/nemo-mbridge-perf-sequence-packing/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-sequence-packing/SKILL.md) +- [skills/nemo-mbridge-perf-sequence-packing/card.yaml](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-sequence-packing/card.yaml) diff --git a/docs/parallelisms.md b/docs/training/parallelisms.mdx similarity index 96% rename from docs/parallelisms.md rename to docs/training/parallelisms.mdx index 6b2124fe2b..83e7af6963 100644 --- a/docs/parallelisms.md +++ b/docs/training/parallelisms.mdx @@ -1,3 +1,8 @@ +--- +title: "Parallelisms Guide" +description: "" +--- + # Parallelisms Guide Megatron Bridge supports various data-parallel and model-parallel deep learning workload deployment methods, which can be mixed together arbitrarily. These parallelism strategies are configured through model provider classes and leverage Megatron Core's implementation for performance and memory efficiency. @@ -21,7 +26,7 @@ Distributed Data Parallelism (DDP) keeps the model copies consistent by synchron In Megatron Bridge, DDP is the default parallel deployment method. The total number of GPUs corresponds to the size of the DP group, and training an LLM with model parallelism decreases the size of the DP group. -To enable the distributed optimizer, configure the {py:class}`bridge.training.config.OptimizerConfig` and {py:class}`bridge.training.config.DistributedDataParallelConfig` +To enable the distributed optimizer, configure the `bridge.training.config.OptimizerConfig` and `bridge.training.config.DistributedDataParallelConfig` ```python from megatron.bridge.training.config import ConfigContainer, DistributedDataParallelConfig, OptimizerConfig @@ -44,7 +49,7 @@ config = ConfigContainer( ) ``` -For more optimizer options, refer to the {py:class}`bridge.training.config.OptimizerConfig` API documentation. +For more optimizer options, refer to the `bridge.training.config.OptimizerConfig` API documentation. ## Model Parallelism @@ -480,7 +485,7 @@ the primary scaling dimension. For operational details on configuring combined parallelism, troubleshooting layouts, and memory estimation, see the -[parallelism strategies skill](skills/nemo-mbridge-perf-parallelism-strategies/SKILL.md). +[parallelism strategies skill](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-parallelism-strategies/SKILL.md). ## Configuration Guidelines @@ -507,8 +512,8 @@ layouts, and memory estimation, see the ## Related Artifacts -- **Operational skill**: [skills/nemo-mbridge-perf-parallelism-strategies/SKILL.md](skills/nemo-mbridge-perf-parallelism-strategies/SKILL.md) โ€” enablement, pitfalls, memory estimation, verification -- **Knowledge card**: [skills/nemo-mbridge-perf-parallelism-strategies/card.yaml](skills/nemo-mbridge-perf-parallelism-strategies/card.yaml) โ€” structured metadata and validation status +- **Operational skill**: [skills/nemo-mbridge-perf-parallelism-strategies/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-parallelism-strategies/SKILL.md) โ€” enablement, pitfalls, memory estimation, verification +- **Knowledge card**: [skills/nemo-mbridge-perf-parallelism-strategies/card.yaml](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-perf-parallelism-strategies/card.yaml) โ€” structured metadata and validation status ## Resources diff --git a/docs/training/peft.md b/docs/training/peft.mdx similarity index 97% rename from docs/training/peft.md rename to docs/training/peft.mdx index 9cfc920b45..293aba599a 100644 --- a/docs/training/peft.md +++ b/docs/training/peft.mdx @@ -1,3 +1,8 @@ +--- +title: "Parameter-Efficient Fine-Tuning (PEFT)" +description: "" +--- + # Parameter-Efficient Fine-Tuning (PEFT) This guide explains how to configure and use PEFT in Megatron Bridgeโ€”covering LoRA and DoRA, required checkpoints, example configurations, and the internal design and training workflowโ€”so you can integrate, scale, and checkpoint adapters efficiently. @@ -36,11 +41,11 @@ config = ConfigContainer( ) ``` -```{note} + **Requirement:** PEFT requires `checkpoint.pretrained_checkpoint` to load the frozen base model weights before adapters are inserted. This path may be a native Megatron checkpoint base directory, a specific native Megatron `iter_N` directory, or a local Hugging Face full-model directory containing `config.json` and model weight files. A remote Hugging Face model ID is not a checkpoint path; download it locally or convert it to Megatron format first. When resuming adapter training, keep `checkpoint.pretrained_checkpoint` pointed at the same base model and set `checkpoint.load` to the PEFT adapter checkpoint directory. -``` + ## Supported PEFT Methods @@ -138,24 +143,17 @@ The distinction comes from the fact that Megatron Core optimizes the implementat The following two figures illustrate the difference between canonical and performant LoRA, using the `linear_qkv` layer as an example. Canonical LoRA runs three adapters sequentially, while performant LoRA runs one adapter. -```{image} images/canonical_lora.png -:width: 640 -:align: center -``` +Canonical Lora -```{image} images/performant_lora.png -:width: 400 -:align: center -``` +Performant Lora Canonical LoRA conforms more closely to reference implementations, though it is slower in comparison since it performs several matrix multiplications sequentially, as described above. Performant LoRA has fewer parameters than canonical LoRA and can often achieve the same level of accuracy as canonical LoRA. Though not immediately apparent, performant LoRA is mathematically equivalent to canonical LoRA when the $A_q$, $A_k$, $A_v$ matrices are tied (i.e. forced to share the same weight during training) in `linear_qkv`, and similarly when the $A_{up}$, $A_{gate}$ matrices are tied in `linear_fc1`. -```{admonition} Mathematical Proof: Performant LoRA Equivalence to Canonical LoRA with Tied Weights -:class: dropdown + -Let $[x \quad y]$ denote matrix concatenation. (In Megatron Bridge, this concatenation is done in an interleaved fashion, but this does not affect the proof below.) +Mathematical Proof: Performant LoRA Equivalence to Canonical LoRA with Tied Weights Let $A_q = A_k = A_v = A_{qkv}$ (weight tying) @@ -189,7 +187,7 @@ Where: - $d$: Transformer head dimension (`kv_channels`). - $r$: LoRA rank. -``` + #### Using Canonical LoRA @@ -240,9 +238,9 @@ The following table lists specific submodules within transformer architectures t | `linear_fc1_gate` | Gate projection in MLP | | `linear_fc2` | Second MLP layer | -```{note} + Canonical LoRA does not support `linear_qkv` or `linear_fc1` targets. Use the individual component targets (`linear_q`, `linear_k`, `linear_v` for QKV and `linear_fc1_up`, `linear_fc1_gate` for FC1) instead. -``` + ### [DoRA: Weight-Decomposed Low-Rank Adaptation](https://arxiv.org/abs/2402.09353) @@ -335,7 +333,7 @@ This section describes the internal design and architecture for how PEFT is inte The PEFT framework introduces a modular design for integrating adapters into large-scale models. Its architecture consists of the following components: -1. **Base PEFT Class**: All PEFT methods inherit from the abstract {py:class}`bridge.peft.base.PEFT` base class, which defines the core interface for module transformation. +1. **Base PEFT Class**: All PEFT methods inherit from the abstract `bridge.peft.base.PEFT` base class, which defines the core interface for module transformation. 2. **Module Transformation**: PEFT traverses the model structure to identify and transform target modules individually. 3. **Adapter Integration**: Adapters are injected into selected modules using a pre-wrap hook during model initialization. 4. **Checkpoint Integration**: Only adapter parameters are saved and loaded during checkpointing; base model weights remain frozen and unchanged. diff --git a/docs/performance-guide.md b/docs/training/performance-tuning.mdx similarity index 99% rename from docs/performance-guide.md rename to docs/training/performance-tuning.mdx index f57f1b4548..e36b00d8be 100644 --- a/docs/performance-guide.md +++ b/docs/training/performance-tuning.mdx @@ -1,14 +1,19 @@ +--- +title: "Performance Tuning Guide" +description: "" +--- + # Performance Tuning Guide Megatron-Bridge provides a wide range of features for performant and memory-efficient LLM training on GPUs, and comes pre-configured with optimal settings. However, factors such as model architecture, hyperparameters, GPU count, and GPU type can affect the available options, and additional tuning may be necessary to achieve optimal performance. This document explores the factors that affect training performance, highlights common issues, and outlines techniques for performance tuning that lead to higher MFU (Model FLOPS Utilization) and TCO. -```{Note} -This guide makes references to several configuration settings. These settings will be referenced relative to the the config class that contains them, e.g. `OptimizerConfig.lr`. Please see for more details on configuration settings. -``` + +This guide makes references to several configuration settings. These settings will be referenced relative to the the config class that contains them, e.g. `OptimizerConfig.lr`. Please see [API reference](/reference/api) for more details on configuration settings. + -```{Note} + This guide references several configuration settings from `TransformerConfig`. Please apply these to the appropriate ModelProvider for your model, e.g. `GPTModelProvider`, as the `ConfigContainer` does not accept a raw `TransformerConfig`. -``` + ## Low Precision Training @@ -299,7 +304,7 @@ Additionally, because CP shards activations, it also partitions optimizer states > 1. Megatron-Bridge supports graph capture, significantly reducing host overhead. CUDA Graph is applicable only to LLMs with a static tensor shape across training steps. For example, it supports fixed-size packed sequences but does not handle sequences with varying lengths at each step. Also, MoE models with token-dropless propagation have limited CUDA graph support, restricted to the dense modules only. > 2. CUDA graph requires additional memory for static buffer management, typically adding a few gigabytes for static buffers, while models with PP size > 1 may consume over 10GB. We are actively working to reduce this memory overhead. - > 3. See [CUDA Graphs](training/cuda-graphs.md) for configuration details (`cuda_graph_impl`, `cuda_graph_scope`). + > 3. See [CUDA Graphs](/training/cuda-graphs) for configuration details (`cuda_graph_impl`, `cuda_graph_scope`). 5. Bind CPU memory for GPU processes @@ -754,7 +759,7 @@ uv run python -u /home/dpsk_a2a/deepep/tests/test_internode.py - `TransformerConfig.cpu_offloading_num_layers` - `TransformerConfig.cpu_offloading_weights` - `GPTProvider.cross_entropy_loss_fusion` -- `TransformerConfig.cuda_graph_impl` / `cuda_graph_scope` (see [CUDA Graphs](training/cuda-graphs.md)) +- `TransformerConfig.cuda_graph_impl` / `cuda_graph_scope` (see [CUDA Graphs](/training/cuda-graphs)) - `MixedPrecisionConfig.fp8_param_gather` - `GPTProvider.gradient_accumulation_fusion` - `TransformerConfig.masked_softmax_fusion` diff --git a/docs/training/profiling.md b/docs/training/profiling.mdx similarity index 90% rename from docs/training/profiling.md rename to docs/training/profiling.mdx index 1520263cbf..a8e8130130 100644 --- a/docs/training/profiling.md +++ b/docs/training/profiling.mdx @@ -1,10 +1,15 @@ +--- +title: "Profiling" +description: "" +--- + # Profiling Megatron Bridge provides built-in support for profiling training jobs using a range of performance analysis tools. These include NVIDIA Nsight Systems (Nsys) for workflow optimization, as well as PyTorch-based profilers and memory trackers to monitor performance and memory usage patterns during training. ## ProfilingConfig Overview -{py:class}`bridge.training.config.ProfilingConfig` is a dataclass that encapsulates profiling-related settings for training. It resides inside the overall {py:class}`bridge.training.config.ConfigContainer`, which represents the complete configuration for a training run. +`bridge.training.config.ProfilingConfig` is a dataclass that encapsulates profiling-related settings for training. It resides inside the overall `bridge.training.config.ConfigContainer`, which represents the complete configuration for a training run. ### Profiling Options @@ -36,9 +41,9 @@ The configuration includes options for recording tensor shapes (`record_shapes`) NVIDIA Nsys is a system-wide performance analysis tool designed to help you tune and optimize CUDA applications. Megatron Bridge integrates with Nsys to enable profiling specific steps of your training job, making it easy to collect detailed performance data without manual instrumentation. -```{note} + NSys profiling cannot be used with the `FaultTolerancePlugin` due to implementation conflicts. If both are enabled, the framework will automatically disable NSys profiling and emit a warning. -``` + ### Configure NSys Profiling @@ -69,7 +74,7 @@ Replace `` with your desired output path and ` ### Configure Profiling with the NeMo Run NSys Plugin -Recipe users can leverage the {py:class}`bridge.recipes.run_plugins.NsysPlugin` to configure NSys profiling through NeMo Run executors. The plugin provides a convenient interface for setting up profiling without manually configuring the underlying NSys command. +Recipe users can leverage the `bridge.recipes.run_plugins.NsysPlugin` to configure NSys profiling through NeMo Run executors. The plugin provides a convenient interface for setting up profiling without manually configuring the underlying NSys command. ```python import nemo_run as run @@ -125,7 +130,7 @@ cfg.profiling = ProfilingConfig( ### Configure Profiling with the PyTorch Profiler Plugin -Similar to NSys, recipe users can use the {py:class}`bridge.recipes.run_plugins.PyTorchProfilerPlugin` for convenient configuration: +Similar to NSys, recipe users can use the `bridge.recipes.run_plugins.PyTorchProfilerPlugin` for convenient configuration: ```python from megatron.bridge.recipes.run_plugins import PyTorchProfilerPlugin diff --git a/docs/recipe-usage.md b/docs/training/recipe-usage.mdx similarity index 96% rename from docs/recipe-usage.md rename to docs/training/recipe-usage.mdx index 4d141f2cfb..d38d81d047 100644 --- a/docs/recipe-usage.md +++ b/docs/training/recipe-usage.mdx @@ -1,13 +1,18 @@ +--- +title: "Using Recipes" +description: "" +--- + # Using Recipes -Megatron Bridge provides production-ready training recipes for several popular models. You can find an overview of supported recipes and ๐Ÿค— Hugging Face bridges [here](index.md#supported-models). +Megatron Bridge provides production-ready training recipes for several popular models. You can find an overview of supported recipes and ๐Ÿค— Hugging Face bridges [here](/#supported-models). This guide will cover the next steps to make use of a training recipe, including how to [override configuration](#overriding-configuration) and how to [launch a job](#launch-methods). ## Overview - **Coverage**: We provide recipes across select model families and sizes, including Llama, Qwen, DeepSeek, and Nemotron-H (Mamba-based). - **Defaults**: Each recipe sets defaults meant for convergence and performance across parallelisms, precision data types, and optimizer & scheduler choices. These recipes can be used as a high-quality starting point. -- **Integration**: Recipes return a single `ConfigContainer` that plugs directly into our training [entry points](training/entry-points.md) (see the published docs as well: https://docs.nvidia.com/nemo/megatron-bridge/latest/training/entry-points.html). +- **Integration**: Recipes return a single `ConfigContainer` that plugs directly into our training [entry points](/training/entry-points) (see the published docs as well: https://docs.nvidia.com/nemo/megatron-bridge/latest/training/entry-points.html). - **Customization**: You can override any part of the recipe (Python, YAML, CLI) to adapt to your data, scale, and objectives. ## Choosing a recipe or a new config @@ -29,7 +34,7 @@ For dataset fields, prefer `seq_length` in Bridge examples. LLM pretraining uses ## Overriding configuration -Recipes are provided through a {py:class}`~bridge.training.config.ConfigContainer` object. This is a dataclass that holds all configuration objects needed for training. You can find a more detailed overview of the `ConfigContainer` [here](training/config-container-overview.md). +Recipes are provided through a `bridge.training.config.ConfigContainer` object. This is a dataclass that holds all configuration objects needed for training. You can find a more detailed overview of the `ConfigContainer` [here](/training/config-container-overview). The benefit of providing the full recipe through a pythonic structure is that it is agnostic to any configuration approach that a user may prefer, whether that's YAML, `argparse` or something else. In other words, the user may override the recipe however they see fit. The following sections detail a few different ways to override the configuration recipe. For a generic recipe launcher, see [`scripts/training/run_recipe.py`](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/scripts/training/run_recipe.py). diff --git a/docs/training/resiliency.md b/docs/training/resiliency.mdx similarity index 90% rename from docs/training/resiliency.md rename to docs/training/resiliency.mdx index 002cf00960..89dabc0a68 100644 --- a/docs/training/resiliency.md +++ b/docs/training/resiliency.mdx @@ -1,3 +1,8 @@ +--- +title: "Resiliency" +description: "" +--- + # Resiliency Megatron Bridge incorporates resilient training features from the @@ -7,7 +12,7 @@ due to failures and interruptions during training. This page is the stable overview for what each resiliency feature is, when to use it, and which constraints are durable. For operational setup, config knobs, -parameter tables, code anchors, and verification commands, see [skills/nemo-mbridge-resiliency/SKILL.md](../skills/nemo-mbridge-resiliency/SKILL.md). +parameter tables, code anchors, and verification commands, see [skills/nemo-mbridge-resiliency/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-resiliency/SKILL.md). ## What It Is @@ -124,7 +129,7 @@ Local checkpointing is useful when: - Node-local storage must have sufficient capacity for at least one checkpoint - Replication degree must be configured to survive the expected failure rate -- Requires compatible checkpoint format (see [skills/nemo-mbridge-resiliency/SKILL.md](../skills/nemo-mbridge-resiliency/SKILL.md)) +- Requires compatible checkpoint format (see [skills/nemo-mbridge-resiliency/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-resiliency/SKILL.md)) ## Re-run State Machine @@ -184,9 +189,9 @@ with job-level fault tolerance. ## Related Docs -- [docs/training/checkpointing.md](checkpointing.md) -- [docs/performance-guide.md](../performance-guide.md) -- [skills/nemo-mbridge-resiliency/SKILL.md](../skills/nemo-mbridge-resiliency/SKILL.md) -- [skills/nemo-mbridge-resiliency/card.yaml](../skills/nemo-mbridge-resiliency/card.yaml) +- [docs/training/checkpointing.mdx](/training/checkpointing) +- [docs/training/performance-tuning.mdx](/training/performance-tuning) +- [skills/nemo-mbridge-resiliency/SKILL.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-resiliency/SKILL.md) +- [skills/nemo-mbridge-resiliency/card.yaml](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/skills/nemo-mbridge-resiliency/card.yaml) - [NVIDIA Resiliency Extension](https://github.com/NVIDIA/nvidia-resiliency-ext) - [In-Process Restart Guide](https://nvidia.github.io/nvidia-resiliency-ext/inprocess/index.html) diff --git a/docs/training/training-loop-settings.md b/docs/training/training-loop-settings.mdx similarity index 93% rename from docs/training/training-loop-settings.md rename to docs/training/training-loop-settings.mdx index c07dac6c90..fa9c93957f 100644 --- a/docs/training/training-loop-settings.md +++ b/docs/training/training-loop-settings.mdx @@ -1,6 +1,11 @@ +--- +title: "Training Loop Configuration" +description: "" +--- + # Training Loop Configuration -The {py:class}`bridge.training.config.TrainingConfig` contains settings related to the training loop bounds, exit conditions, validation, batch sizing, and memory management. +The `bridge.training.config.TrainingConfig` contains settings related to the training loop bounds, exit conditions, validation, batch sizing, and memory management. ## Key Parameters @@ -42,7 +47,7 @@ Megatron-Bridge supports two modes for specifying training duration: - You must specify **exactly one** of `train_iters` or `train_samples` - not both. - When using `train_samples`, training iterations are automatically calculated as `train_samples // global_batch_size`. - Batch size rampup (`rampup_batch_size`) is not currently supported with sample-based training. -- Your scheduler configuration should match your training mode (see [Learning Rate Scheduling](optimizer-scheduler.md#learning-rate-scheduling)). +- Your scheduler configuration should match your training mode (see [Learning Rate Scheduling](/training/optimizer-scheduler#learning-rate-scheduling)). ### Validation Configure validation frequency, duration, and evaluation-only modes. @@ -83,4 +88,3 @@ Monitor training consistency and synchronization across distributed processes. |-----------|------|---------|-------------| | `check_weight_hash_across_dp_replicas_interval` | `Optional[int]` | `None` | Check weight hash consistency across data parallel replicas | | `train_sync_interval` | `Optional[int]` | `None` | CPU-GPU synchronization interval to prevent CPU running ahead | - diff --git a/docs/troubleshooting/index.mdx b/docs/troubleshooting/index.mdx new file mode 100644 index 0000000000..0c3bdd04fe --- /dev/null +++ b/docs/troubleshooting/index.mdx @@ -0,0 +1,37 @@ +--- +title: "Troubleshooting" +description: "Troubleshooting entry point for Megatron Bridge setup, runtime, and known issue guidance." +--- + +# Troubleshooting + +Use this section to diagnose setup problems, runtime failures, known release issues, and training performance regressions. + +## Quick Checks + +1. Confirm that the correct container or Python environment is active. +2. Confirm that the expected Hugging Face token is available when using gated models. +3. Confirm that `3rdparty/Megatron-LM` is initialized and on the expected branch. +4. Review model-family notes for required patches, dependencies, and validated parallelism settings. + +## Troubleshooting Topics + + + + +Review known release issues and workarounds. + + + +Check dependency and container compatibility. + + + +Review common performance issues and tuning guidance. + + + +Debug checkpoint save, load, format, and resume behavior. + + + diff --git a/docs/releases/known-issues.md b/docs/troubleshooting/known-issues.mdx similarity index 97% rename from docs/releases/known-issues.md rename to docs/troubleshooting/known-issues.mdx index 992f53e9ab..9325f6fc30 100644 --- a/docs/releases/known-issues.md +++ b/docs/troubleshooting/known-issues.mdx @@ -1,3 +1,8 @@ +--- +title: "Known Issues" +description: "" +--- + # Known Issues This page lists known issues and limitations in the current release. diff --git a/docs/use-cases/index.mdx b/docs/use-cases/index.mdx new file mode 100644 index 0000000000..1527b28854 --- /dev/null +++ b/docs/use-cases/index.mdx @@ -0,0 +1,20 @@ +--- +title: "Use Cases" +description: "Use-case-oriented Megatron Bridge documentation." +--- + +# Use Cases + +Use-case pages organize procedural and integration guidance that does not fit a single model-family or reference page. + + + + +Add support for new model families. + + + +Use Megatron Bridge with reinforcement learning workflows. + + + diff --git a/docs/adding-new-models.md b/docs/use-cases/model-development/add-new-model.mdx similarity index 88% rename from docs/adding-new-models.md rename to docs/use-cases/model-development/add-new-model.mdx index 0510fb8260..c1363fe9f7 100644 --- a/docs/adding-new-models.md +++ b/docs/use-cases/model-development/add-new-model.mdx @@ -1,8 +1,13 @@ +--- +title: "Contribute a New Model to Megatron Bridge" +description: "" +--- + # Contribute a New Model to Megatron Bridge -This guide explains how to add support for a new ๐Ÿค— Hugging Face model (or family) to Megatron Bridge so to convert between HF โ†” Megatron-Core formats and participate in training recipes. +This guide explains how to add support for a new ๐Ÿค— Hugging Face model (or family) to Megatron Bridge so to convert between HF to and from Megatron-Core formats and participate in training recipes. -Use this checklist-style flow: scaffold โ†’ provider mapping โ†’ parameter mappings โ†’ tests โ†’ validation. +Use this checklist-style flow: scaffold -> provider mapping -> parameter mappings -> tests -> validation. ## Prerequisites @@ -12,8 +17,8 @@ Use this checklist-style flow: scaffold โ†’ provider mapping โ†’ parameter mappi - Familiarity with Megatron-Core GPT-style modules and ๐Ÿค— Transformers config objects. - Access to a small HF checkpoint for local testing. - Read first: - - [Bridge user guide](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/docs/bridge-guide.md) - - [Technical details](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/docs/bridge-tech-details.md) + - [Bridge user guide](/get-started/quickstart/hugging-face-conversion) + - [Technical details](/about/architecture) - [Model bridges overview](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/models/README.md) @@ -30,7 +35,7 @@ Use this checklist-style flow: scaffold โ†’ provider mapping โ†’ parameter mappi Create a folder under `src/megatron/bridge/models//` and add: - `_provider.py`: builds a `TransformerConfig`-compatible provider (or a subclass of an existing provider) and exposes `.provide_distributed_model()`. For example: [Llama provider](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/models/llama/llama_provider.py), [Qwen provider](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/models/qwen/qwen_provider.py), or [Qwen2 provider](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/models/qwen/qwen2_provider.py). -- `_bridge.py`: architecture-specific bridge that maps HF config โ†’ provider and defines parameter mappings. For example: [Llama bridge](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/models/llama/llama_bridge.py), [Qwen3 bridge](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/models/qwen/qwen3_bridge.py), or [Qwen2 bridge](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/models/qwen/qwen2_bridge.py). +- `_bridge.py`: architecture-specific bridge that maps HF config -> provider and defines parameter mappings. For example: [Llama bridge](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/models/llama/llama_bridge.py), [Qwen3 bridge](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/models/qwen/qwen3_bridge.py), or [Qwen2 bridge](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/models/qwen/qwen2_bridge.py). - Optional: `README.md` with any model quirks. For example: [Llama README](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/models/llama/README.md). ## 3) Implement the Provider @@ -53,7 +58,7 @@ model = provider.provide_distributed_model(wrap_with_ddp=False) ```text You are working in the Megatron Bridge repo. Create `src/megatron/bridge/models//_provider.py`. -Goal: Implement `YourModelProvider` that maps HF config โ†’ Megatron-Core transformer config and exposes `.provide_distributed_model()`. +Goal: Implement `YourModelProvider` that maps HF config -> Megatron-Core transformer config and exposes `.provide_distributed_model()`. Requirements: - Start from `src/megatron/bridge/models/gpt_provider.py` and adapt. @@ -80,7 +85,7 @@ Use the `provider_bridge` method to map Hugging Face configs to a Megatron model - `provider_bridge`: see [model_bridge.py](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/models/conversion/model_bridge.py) - `MegatronMappingRegistry`: see [mapping_registry.py](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/models/conversion/mapping_registry.py) - Mapping implementations: see [param_mapping.py](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/models/conversion/param_mapping.py) -- Background: see [Bridge technical details](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/docs/bridge-tech-details.md) +- Background: see [Bridge technical details](/about/architecture) Example registration skeleton: @@ -174,8 +179,8 @@ References: - `src/megatron/bridge/models/qwen/qwen2_bridge.py` Acceptance: -- HF โ†’ Megatron load completes with no missing parameters (for a tiny model). -- Megatron โ†’ HF export returns tensors with expected shapes/dtypes for several keys. +- HF -> Megatron load completes with no missing parameters (for a tiny model). +- Megatron -> HF export returns tensors with expected shapes/dtypes for several keys. ``` ## 5) Minimal smoke test (local) @@ -184,7 +189,7 @@ A minimal bidirectional end-to-end check: ```python from megatron.bridge import AutoBridge -# HF โ†’ Megatron +# HF -> Megatron bridge = AutoBridge.from_hf_pretrained("/", trust_remote_code=True) provider = bridge.to_megatron_provider() provider.tensor_model_parallel_size = 1 @@ -193,7 +198,7 @@ provider.finalize() model = provider.provide_distributed_model(wrap_with_ddp=False) bridge.load_hf_weights(model) -# Megatron โ†’ HF (stream a few tensors) +# Megatron -> HF (stream a few tensors) for i, (name, tensor) in enumerate(bridge.export_hf_weights(model, cpu=True)): print(name, tuple(tensor.shape)) if i > 10: @@ -219,8 +224,8 @@ Add or extend tests under `tests/functional_tests/models//` and `tes Tests are organized in model-specific subdirectories that mirror the source structure in `src/megatron/bridge/models/`. - Conversion coverage: - - HF โ†’ Megatron load succeeds without missing params - - Megatron โ†’ HF export round-trips shapes and dtypes + - HF -> Megatron load succeeds without missing params + - Megatron -> HF export round-trips shapes and dtypes - Provider coverage: - Provider fields align with HF config (heads, groups, FFN size, RoPE) - Optional numeric checks: @@ -264,8 +269,8 @@ Create a subdirectory `tests/functional_tests/models//` with an `__i - Assert provider fields match HF config (heads, groups, hidden size, FFN, RoPE, vocab size, max position). 2) `test__conversion.py` - - HF โ†’ Megatron: load HF weights into the Megatron model via the bridge; assert no missing/extra params. - - Megatron โ†’ HF: export a subset of tensors; assert shape/dtype parity with HF. + - HF -> Megatron: load HF weights into the Megatron model via the bridge; assert no missing/extra params. + - Megatron -> HF: export a subset of tensors; assert shape/dtype parity with HF. - Optionally run a short generation on CPU and compare logits numerically within tolerance. Use `tests/functional_tests/models/qwen/test_qwen3_provider.py` and `test_qwen3_conversion.py` as templates. @@ -300,9 +305,8 @@ logging.getLogger("megatron.bridge").setLevel(logging.DEBUG) ## 10) Useful links -- User guide: [docs/bridge-guide.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/docs/bridge-guide.md) -- Technical deep-dive: [docs/bridge-tech-details.md](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/docs/bridge-tech-details.md) +- User guide: [Hugging Face conversion quickstart](/get-started/quickstart/hugging-face-conversion) +- Technical deep-dive: [Architecture](/about/architecture) - Code examples: [examples/conversion/](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/examples/conversion) - Providers and bridges: [src/megatron/bridge/models/](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/src/megatron/bridge/models) - GitHub source tree: [Megatron Bridge src/megatron/bridge](https://github.com/NVIDIA-NeMo/Megatron-Bridge/tree/main/src/megatron/bridge) - diff --git a/docs/use-cases/model-development/index.mdx b/docs/use-cases/model-development/index.mdx new file mode 100644 index 0000000000..1af6f91851 --- /dev/null +++ b/docs/use-cases/model-development/index.mdx @@ -0,0 +1,20 @@ +--- +title: "Model Development" +description: "Guidance for adding and validating model support in Megatron Bridge." +--- + +# Model Development + +Use this section when adding a new Hugging Face model family or extending existing conversion support. + + + + +Implement providers, bridges, parameter mappings, tests, and documentation for a new model. + + + +Review existing model-family patterns before adding a new implementation. + + + diff --git a/docs/use-cases/reinforcement-learning/index.mdx b/docs/use-cases/reinforcement-learning/index.mdx new file mode 100644 index 0000000000..4347bf8651 --- /dev/null +++ b/docs/use-cases/reinforcement-learning/index.mdx @@ -0,0 +1,20 @@ +--- +title: "Reinforcement Learning" +description: "Reinforcement learning integration guidance for Megatron Bridge." +--- + +# Reinforcement Learning + +Use this section for Megatron Bridge integration patterns with reinforcement learning workflows. + + + + +Review the converted reinforcement learning integration guide. + + + +Review conversion primitives used by RL integration workflows. + + + diff --git a/docs/bridge-rl-integration.md b/docs/use-cases/reinforcement-learning/rl-framework-integration.mdx similarity index 97% rename from docs/bridge-rl-integration.md rename to docs/use-cases/reinforcement-learning/rl-framework-integration.mdx index 5dc70fcf9a..87bdd43d3e 100644 --- a/docs/bridge-rl-integration.md +++ b/docs/use-cases/reinforcement-learning/rl-framework-integration.mdx @@ -1,8 +1,13 @@ +--- +title: "Adapting Megatron Bridge in Reinforcement Learning Frameworks" +description: "" +--- + # Adapting Megatron Bridge in Reinforcement Learning Frameworks Megatron Bridge provides a clean, parallelism-aware path to use ๐Ÿค— Hugging Face models with Megatron-Core training and convert back again for inference. This guide shows how to adapt Megatron Bridge into a new RL framework to: -- Convert Hugging Face (HF) checkpoints โ†’ Megatron format for scalable training +- Convert Hugging Face (HF) checkpoints -> Megatron format for scalable training - Train with Megatron-Core using TP/PP/CP/MoE parallelism, checkpointing, and efficient data paths - Refit trained weights back to HF for deployment with inference engines (e.g., vLLM), including zero-copy/IPC flows @@ -26,7 +31,7 @@ export HF_TOKEN= ``` -## 1) One-time HF โ†’ Megatron checkpoint conversion +## 1) One-time HF -> Megatron checkpoint conversion Use `AutoBridge` to import an HF model into Megatron format. This writes a Megatron checkpoint directory with a `run_config.yaml` you will reuse during training. @@ -206,7 +211,7 @@ forward_backward = get_forward_backward_func() # Your loss function should return (loss_tensor, metrics_dict) def rl_loss_fn(outputs, batch): - # Compute logits โ†’ loss for your RL objective (e.g., PPO, DPO) + # Compute logits -> loss for your RL objective (e.g., PPO, DPO) loss = outputs.sum() * 0.0 # placeholder return loss, {"loss": loss.detach()} @@ -258,7 +263,7 @@ def get_token_logprobs(model, batch): input_ids = batch["input_ids"].to("cuda") outputs = model(input_ids=input_ids) - # Reduce TP logits โ†’ local logprobs for targets + # Reduce TP logits -> local logprobs for targets tp_group = get_tensor_model_parallel_group() tp_rank = get_tensor_model_parallel_rank() @@ -306,7 +311,7 @@ Tips: - Prefer fully-parallel save/load at scale (`fully_parallel_save=True`, `fully_parallel_load=True`). -## 6) Refit: Megatron โ†’ HF for inference (vLLM, Triton, etc.) +## 6) Refit: Megatron -> HF for inference (vLLM, Triton, etc.) Two common pathways: @@ -501,6 +506,6 @@ class MegatronBridgeAdapter: ## See also -- [Bridge with ๐Ÿค— Hugging Face](./bridge-guide.md) for HFโ†”Megatron conversion overview +- [Bridge with ๐Ÿค— Hugging Face](/get-started/quickstart/hugging-face-conversion) for HF to and from Megatron conversion overview - [nemo_rl/models/megatron/community_import.py](https://github.com/NVIDIA-NeMo/RL/blob/main/nemo_rl/models/megatron/community_import.py) for import/export helpers - [nemo_rl/models/policy/megatron_policy_worker.py](https://github.com/NVIDIA-NeMo/RL/blob/main/nemo_rl/models/policy/megatron_policy_worker.py) for end-to-end RL integration (training, logprobs, generation, refit) diff --git a/docs/versions1.json b/docs/versions1.json deleted file mode 100644 index 77f566734d..0000000000 --- a/docs/versions1.json +++ /dev/null @@ -1,37 +0,0 @@ -[ - { - "version": "nightly", - "url": "https://docs.nvidia.com/nemo/megatron-bridge/nightly/" - }, - { - "name": "0.4.2 (latest) ยท 26.04.01", - "version": "0.4.2", - "url": "https://docs.nvidia.com/nemo/megatron-bridge/latest/", - "preferred": true - }, - { - "name": "0.4.0 ยท 26.04", - "version": "0.4.0", - "url": "https://docs.nvidia.com/nemo/megatron-bridge/0.4.0/" - }, - { - "name": "0.3.1 ยท 26.02.01", - "version": "0.3.1", - "url": "https://docs.nvidia.com/nemo/megatron-bridge/0.3.1/" - }, - { - "name": "0.3.0 ยท 26.02", - "version": "0.3.0", - "url": "https://docs.nvidia.com/nemo/megatron-bridge/0.3.0/" - }, - { - "name": "0.2.0 ยท 25.11", - "version": "0.2.0", - "url": "https://docs.nvidia.com/nemo/megatron-bridge/0.2.0/" - }, - { - "name": "0.1.0 ยท 25.09", - "version": "0.1.0", - "url": "https://docs.nvidia.com/nemo/megatron-bridge/0.1.0/" - } -] diff --git a/examples/models/deepseek_v4/README.md b/examples/models/deepseek_v4/README.md index d36822a737..d016fac470 100644 --- a/examples/models/deepseek_v4/README.md +++ b/examples/models/deepseek_v4/README.md @@ -2,7 +2,7 @@ End-to-end conversion and inference scripts for the DeepSeek V4 family on Megatron Bridge. -The bridge supports four published variants out of the same code path. The on-disk quantisation differs between post-trained (Flash, Pro) and pretrained-only (Flash-Base, Pro-Base) models โ€” see [`docs/models/deepseek/deepseek-v4.md`](../../../docs/models/deepseek/deepseek-v4.md) for the per-variant scheme. +The bridge supports four published variants out of the same code path. The on-disk quantisation differs between post-trained (Flash, Pro) and pretrained-only (Flash-Base, Pro-Base) models โ€” see [`docs/models/deepseek/deepseek-v4.md`](../../../docs/models/deepseek/deepseek-v4.mdx) for the per-variant scheme. ## MCore Checkout diff --git a/examples/models/gemma/gemma3_vl/README.md b/examples/models/gemma/gemma3_vl/README.md index f89074fc42..b60e46e9ff 100644 --- a/examples/models/gemma/gemma3_vl/README.md +++ b/examples/models/gemma/gemma3_vl/README.md @@ -2,7 +2,7 @@ This directory contains example scripts for Gemma 3 VL vision-language models. -For model introduction and architecture details, see the [Gemma 3 VL documentation](../../../../docs/models/gemma/gemma3-vl.md). +For model introduction and architecture details, see the [Gemma 3 VL documentation](../../../../docs/models/gemma/gemma3-vl.mdx). ## Workspace Configuration @@ -83,7 +83,7 @@ The image is a table comparing the technical specifications of two ## Finetune Recipes -- See: [bridge.recipes.gemma3_vl](../../../../docs/apidocs/bridge/bridge.recipes.gemma3_vl.md) +- See: [bridge.recipes.gemma3_vl](../../../../docs/reference/api/index.mdx) - Available recipes: - `gemma3_vl_4b_finetune_config`: Finetuning for 4B VL model with PEFT support - `gemma3_vl_12b_finetune_config`: Finetuning for 12B VL model with PEFT support diff --git a/examples/models/glm/glm_45v/README.md b/examples/models/glm/glm_45v/README.md index 9263bf1285..b65e403c4f 100644 --- a/examples/models/glm/glm_45v/README.md +++ b/examples/models/glm/glm_45v/README.md @@ -2,7 +2,7 @@ This directory contains example scripts for GLM-4.5V vision-language model. -For model introduction and architecture details, see the [GLM-4.5V documentation](../../../../docs/models/glm/glm-45v.md). +For model introduction and architecture details, see the [GLM-4.5V documentation](../../../../docs/models/glm/glm-45v.mdx). ## Workspace Configuration @@ -84,7 +84,7 @@ Here's a breakdown of the information presented: ## Finetune Recipes -- See: [bridge.recipes.glm_vl](../../../../docs/apidocs/bridge/bridge.recipes.glm_vl.md) +- See: [bridge.recipes.glm_vl](../../../../docs/reference/api/index.mdx) - Available recipes: - `glm_45v_sft_config`: Full supervised fine-tuning for GLM-4.5V - `glm_45v_peft_config`: Parameter-efficient fine-tuning for GLM-4.5V with LoRA or DoRA diff --git a/examples/models/mistral/ministral3/README.md b/examples/models/mistral/ministral3/README.md index 071841be99..326674a7e6 100644 --- a/examples/models/mistral/ministral3/README.md +++ b/examples/models/mistral/ministral3/README.md @@ -2,7 +2,7 @@ This directory contains example scripts for Ministral 3 vision-language models. -For model introduction and architecture details, see the [Ministral 3 documentation](../../../../docs/models/mistral/ministral3.md). +For model introduction and architecture details, see the [Ministral 3 documentation](../../../../docs/models/mistral/ministral3.mdx). ## Workspace Configuration @@ -83,7 +83,7 @@ The current date is {today}. ## Finetune Recipes -- See: [bridge.recipes.ministral3](../../../../docs/apidocs/bridge/bridge.recipes.ministral3.md) +- See: [bridge.recipes.ministral3](../../../../docs/reference/api/index.mdx) - Available recipes: - `ministral3_3b_sft_config`: Finetuning for 3B VL model - `ministral3_8b_sft_config`: Finetuning for 8B VL model diff --git a/examples/models/qwen/qwen35_vl/README.md b/examples/models/qwen/qwen35_vl/README.md index b22ea34ce5..23cb91983c 100644 --- a/examples/models/qwen/qwen35_vl/README.md +++ b/examples/models/qwen/qwen35_vl/README.md @@ -2,7 +2,7 @@ This directory contains example scripts for Qwen3.5-VL vision-language models. -For model introduction and architecture details, see the [Qwen3.5-VL documentation](../../../../docs/models/qwen/qwen35-vl.md). +For model introduction and architecture details, see the [Qwen3.5-VL documentation](../../../../docs/models/qwen/qwen35-vl.mdx). ## Workspace Configuration diff --git a/examples/models/qwen/qwen3_omni/README.md b/examples/models/qwen/qwen3_omni/README.md index 03d7f8f0f5..06618a5e47 100644 --- a/examples/models/qwen/qwen3_omni/README.md +++ b/examples/models/qwen/qwen3_omni/README.md @@ -2,7 +2,7 @@ This directory contains example scripts for **Qwen3-Omni thinker-side support** in Megatron Bridge. -For model introduction and implementation notes, see the [Qwen3-Omni documentation](../../../../docs/models/qwen/qwen3-omni.md). +For model introduction and implementation notes, see the [Qwen3-Omni documentation](../../../../docs/models/qwen/qwen3-omni.mdx). ## Current Scope diff --git a/examples/models/qwen/qwen3_vl/README.md b/examples/models/qwen/qwen3_vl/README.md index 7dcaa9ff22..2aa7d5d0d6 100644 --- a/examples/models/qwen/qwen3_vl/README.md +++ b/examples/models/qwen/qwen3_vl/README.md @@ -2,7 +2,7 @@ This directory contains example scripts for Qwen 3 vision-language models. -For model introduction and architecture details, see the [Qwen 3 - VL documentation](../../../../docs/models/qwen/qwen3-vl.md). +For model introduction and architecture details, see the [Qwen 3 - VL documentation](../../../../docs/models/qwen/qwen3-vl.mdx). ## Workspace Configuration diff --git a/skills/nemo-mbridge-mlm-bridge-training/SKILL.md b/skills/nemo-mbridge-mlm-bridge-training/SKILL.md index c059545e80..59835506ad 100644 --- a/skills/nemo-mbridge-mlm-bridge-training/SKILL.md +++ b/skills/nemo-mbridge-mlm-bridge-training/SKILL.md @@ -9,7 +9,7 @@ when_to_use: Running training, comparing MLM vs Bridge loss curves, translating For how they differ, the arg mapping tables, gotchas, and translation script, see: -- @docs/megatron-lm-to-megatron-bridge.md +- @docs/get-started/migration/megatron-lm-to-megatron-bridge.mdx ## First Answer Checklist @@ -122,7 +122,7 @@ SFT/PEFT variants use `_sft_config` / `_peft_config` suffix. ## Megatron-Core Submodule For what the submodule is and why two versions exist, see -@docs/megatron-lm-to-megatron-bridge.md. +@docs/get-started/migration/megatron-lm-to-megatron-bridge.mdx. ### Check current version diff --git a/skills/nemo-mbridge-mlm-bridge-training/card.yaml b/skills/nemo-mbridge-mlm-bridge-training/card.yaml index 45b1f5554b..fbec7b1a73 100644 --- a/skills/nemo-mbridge-mlm-bridge-training/card.yaml +++ b/skills/nemo-mbridge-mlm-bridge-training/card.yaml @@ -24,7 +24,7 @@ feature_meaning: overrides, launch commands, or standalone recipe files. recommended_path: correlation_testing: vanilla_gpt_pretrain_config - arg_mapping_reference: docs/megatron-lm-to-megatron-bridge.md + arg_mapping_reference: docs/get-started/migration/megatron-lm-to-megatron-bridge.mdx known_constraints: - MLM requires --eval-iters and --eval-interval (no defaults). - Bridge scheduler asserts lr_warmup_iters < lr_decay_iters. @@ -36,7 +36,7 @@ known_limitations: - Model-specific recipes carry their own vocab_size which may not match the tokenizer. - Translation script covers common args but may not handle all edge cases. evidence: - - docs/megatron-lm-to-megatron-bridge.md + - docs/get-started/migration/megatron-lm-to-megatron-bridge.mdx - scripts/training/run_recipe.py - scripts/translate_mlm_to_bridge.py - 3rdparty/Megatron-LM/pretrain_gpt.py diff --git a/skills/nemo-mbridge-mlm-bridge-training/skill-card.md b/skills/nemo-mbridge-mlm-bridge-training/skill-card.md index 3af7a92afb..8da1145945 100644 --- a/skills/nemo-mbridge-mlm-bridge-training/skill-card.md +++ b/skills/nemo-mbridge-mlm-bridge-training/skill-card.md @@ -19,7 +19,7 @@ Risk: Review before execution as proposals could introduce incorrect or misleadi Mitigation: Review and scan skill before deployment.
## Reference(s):
-- [Megatron-LM to Megatron Bridge Guide](docs/megatron-lm-to-megatron-bridge.md)
+- [Megatron-LM to Megatron Bridge Guide](docs/get-started/migration/megatron-lm-to-megatron-bridge.mdx)
- [Megatron Bridge Documentation](https://docs.nvidia.com/nemo/megatron-bridge/latest/)
diff --git a/skills/nemo-mbridge-perf-activation-recompute/SKILL.md b/skills/nemo-mbridge-perf-activation-recompute/SKILL.md index 356473d164..66a4bedbb2 100644 --- a/skills/nemo-mbridge-perf-activation-recompute/SKILL.md +++ b/skills/nemo-mbridge-perf-activation-recompute/SKILL.md @@ -7,7 +7,7 @@ when_to_use: Reducing GPU memory via activation recompute, or investigating a co # Activation Recompute -Stable docs: @docs/training/activation-recomputation.md +Stable docs: @docs/training/activation-recomputation.mdx Card: @skills/nemo-mbridge-perf-activation-recompute/card.yaml ## What It Is diff --git a/skills/nemo-mbridge-perf-activation-recompute/card.yaml b/skills/nemo-mbridge-perf-activation-recompute/card.yaml index f5df90608c..694f347b68 100644 --- a/skills/nemo-mbridge-perf-activation-recompute/card.yaml +++ b/skills/nemo-mbridge-perf-activation-recompute/card.yaml @@ -163,7 +163,7 @@ known_limitations: - no automatic selection of optimal recompute_modules - memory savings from layernorm are negligible on most architectures evidence: - - docs/training/activation-recomputation.md + - docs/training/activation-recomputation.mdx - "PR #3107 (Llama3 70B SFT OOM fix experiment)" - src/megatron/bridge/recipes/llama/llama3.py - 3rdparty/Megatron-LM/megatron/core/transformer/transformer_config.py diff --git a/skills/nemo-mbridge-perf-activation-recompute/skill-card.md b/skills/nemo-mbridge-perf-activation-recompute/skill-card.md index aea60bcdfd..358a32c0d9 100644 --- a/skills/nemo-mbridge-perf-activation-recompute/skill-card.md +++ b/skills/nemo-mbridge-perf-activation-recompute/skill-card.md @@ -19,7 +19,7 @@ Risk: Review before execution as proposals could introduce incorrect or misleadi Mitigation: Review and scan skill before deployment.
## Reference(s):
-- [Performance Tuning Guide](docs/performance-guide.md)
+- [Performance Tuning Guide](docs/training/performance-tuning.mdx)
- [Megatron Bridge Documentation](https://docs.nvidia.com/nemo/megatron-bridge/latest/)
diff --git a/skills/nemo-mbridge-perf-cpu-offloading/SKILL.md b/skills/nemo-mbridge-perf-cpu-offloading/SKILL.md index c8f52df0d1..fb98815c4b 100644 --- a/skills/nemo-mbridge-perf-cpu-offloading/SKILL.md +++ b/skills/nemo-mbridge-perf-cpu-offloading/SKILL.md @@ -9,7 +9,7 @@ when_to_use: Enabling CPU offload to reduce GPU memory, or investigating a commi ## References -- Stable docs: @docs/training/cpu-offloading.md +- Stable docs: @docs/training/cpu-offloading.mdx - Structured metadata: @skills/nemo-mbridge-perf-cpu-offloading/card.yaml ## What It Is diff --git a/skills/nemo-mbridge-perf-cpu-offloading/card.yaml b/skills/nemo-mbridge-perf-cpu-offloading/card.yaml index 5c11b9f4af..ef55460c3b 100644 --- a/skills/nemo-mbridge-perf-cpu-offloading/card.yaml +++ b/skills/nemo-mbridge-perf-cpu-offloading/card.yaml @@ -199,7 +199,7 @@ known_limitations: - Optimizer offloading throughput penalty scales linearly with fraction - D2H/H2D overlap provides only ~7% improvement evidence: - - docs/training/cpu-offloading.md + - docs/training/cpu-offloading.mdx - 3rdparty/Megatron-LM/megatron/core/transformer/transformer_config.py - 3rdparty/Megatron-LM/megatron/core/optimizer/optimizer_config.py - 3rdparty/Megatron-LM/megatron/core/optimizer/__init__.py diff --git a/skills/nemo-mbridge-perf-cpu-offloading/skill-card.md b/skills/nemo-mbridge-perf-cpu-offloading/skill-card.md index bce5fc3ffc..0804000310 100644 --- a/skills/nemo-mbridge-perf-cpu-offloading/skill-card.md +++ b/skills/nemo-mbridge-perf-cpu-offloading/skill-card.md @@ -19,7 +19,7 @@ Risk: Review before execution as proposals could introduce incorrect or misleadi Mitigation: Review and scan skill before deployment.
## Reference(s):
-- [CPU Offloading Documentation](docs/training/cpu-offloading.md)
+- [CPU Offloading Documentation](docs/training/cpu-offloading.mdx)
- [Megatron Bridge Documentation](https://docs.nvidia.com/nemo/megatron-bridge/latest/)
diff --git a/skills/nemo-mbridge-perf-cuda-graphs/SKILL.md b/skills/nemo-mbridge-perf-cuda-graphs/SKILL.md index 327fbc465a..828fcf536b 100644 --- a/skills/nemo-mbridge-perf-cuda-graphs/SKILL.md +++ b/skills/nemo-mbridge-perf-cuda-graphs/SKILL.md @@ -7,7 +7,7 @@ when_to_use: Reducing host-driver overhead via CUDA graphs, or tracing a crash o # CUDA Graphs -Stable documentation: @docs/training/cuda-graphs.md +Stable documentation: @docs/training/cuda-graphs.mdx Card: @skills/nemo-mbridge-perf-cuda-graphs/card.yaml ## What It Is @@ -40,8 +40,8 @@ For recompute-heavy workloads: Related docs: -- @docs/training/cuda-graphs.md -- @docs/training/activation-recomputation.md +- @docs/training/cuda-graphs.mdx +- @docs/training/activation-recomputation.mdx ## Enablement diff --git a/skills/nemo-mbridge-perf-cuda-graphs/card.yaml b/skills/nemo-mbridge-perf-cuda-graphs/card.yaml index 361c4b4d33..205f3fa840 100644 --- a/skills/nemo-mbridge-perf-cuda-graphs/card.yaml +++ b/skills/nemo-mbridge-perf-cuda-graphs/card.yaml @@ -227,7 +227,7 @@ measured_results: memory_at_iter1_reserved_gb: "72-73" jobs: "10111518" minimal_example: - docs_example: docs/training/cuda-graphs.md + docs_example: docs/training/cuda-graphs.mdx skill_example: skills/nemo-mbridge-perf-cuda-graphs/SKILL.md test_path: tests/functional_tests/test_groups/recipes/test_llama_recipes_pretrain_cuda_graphs.py cli_path: scripts/performance/run_script.py @@ -275,8 +275,8 @@ known_limitations: - TE-scoped graphs and packed-sequence finetuning remain a fragile combination across model families. - Large-model MoE rollout is memory-headroom-sensitive even when smaller-model TE-scoped graphs work. evidence: - - docs/training/cuda-graphs.md - - docs/performance-guide.md + - docs/training/cuda-graphs.mdx + - docs/training/performance-tuning.mdx - src/megatron/bridge/training/config.py - src/megatron/bridge/training/train.py - src/megatron/bridge/models/gpt_provider.py diff --git a/skills/nemo-mbridge-perf-cuda-graphs/skill-card.md b/skills/nemo-mbridge-perf-cuda-graphs/skill-card.md index b095eec5fa..978671ca7a 100644 --- a/skills/nemo-mbridge-perf-cuda-graphs/skill-card.md +++ b/skills/nemo-mbridge-perf-cuda-graphs/skill-card.md @@ -19,8 +19,8 @@ Risk: Review before execution as proposals could introduce incorrect or misleadi Mitigation: Review and scan skill before deployment.
## Reference(s):
-- [CUDA Graphs Training Documentation](docs/training/cuda-graphs.md)
-- [Performance Tuning Guide](docs/performance-guide.md)
+- [CUDA Graphs Training Documentation](docs/training/cuda-graphs.mdx)
+- [Performance Tuning Guide](docs/training/performance-tuning.mdx)
## Skill Output:
diff --git a/skills/nemo-mbridge-perf-expert-parallel-overlap/SKILL.md b/skills/nemo-mbridge-perf-expert-parallel-overlap/SKILL.md index b05ac9bffd..fbc1365a41 100644 --- a/skills/nemo-mbridge-perf-expert-parallel-overlap/SKILL.md +++ b/skills/nemo-mbridge-perf-expert-parallel-overlap/SKILL.md @@ -9,7 +9,7 @@ when_to_use: Enabling EP overlap to hide dispatch/combine latency, or tracing a ## References -- Stable docs: @docs/training/communication-overlap.md +- Stable docs: @docs/training/communication-overlap.mdx - Structured metadata: @skills/nemo-mbridge-perf-expert-parallel-overlap/card.yaml ## What It Is diff --git a/skills/nemo-mbridge-perf-expert-parallel-overlap/card.yaml b/skills/nemo-mbridge-perf-expert-parallel-overlap/card.yaml index 6839e8968e..5a4b0daab9 100644 --- a/skills/nemo-mbridge-perf-expert-parallel-overlap/card.yaml +++ b/skills/nemo-mbridge-perf-expert-parallel-overlap/card.yaml @@ -172,7 +172,7 @@ known_limitations: - End-to-end throughput gains not yet measured in a controlled Bridge experiment. evidence: - - docs/training/communication-overlap.md + - docs/training/communication-overlap.mdx - src/megatron/bridge/training/comm_overlap.py - src/megatron/bridge/training/flex_dispatcher_backend.py - src/megatron/bridge/training/config.py diff --git a/skills/nemo-mbridge-perf-expert-parallel-overlap/skill-card.md b/skills/nemo-mbridge-perf-expert-parallel-overlap/skill-card.md index db2b29e0f7..c5f56b2bc1 100644 --- a/skills/nemo-mbridge-perf-expert-parallel-overlap/skill-card.md +++ b/skills/nemo-mbridge-perf-expert-parallel-overlap/skill-card.md @@ -19,8 +19,8 @@ Risk: Review before execution as proposals could introduce incorrect or misleadi Mitigation: Review and scan skill before deployment.
## Reference(s):
-- [Communication Overlap Documentation](docs/training/communication-overlap.md)
-- [Performance Tuning Guide](docs/performance-guide.md)
+- [Communication Overlap Documentation](docs/training/communication-overlap.mdx)
+- [Performance Tuning Guide](docs/training/performance-tuning.mdx)
- [Megatron-Bridge GitHub Repository](https://github.com/NVIDIA-NeMo/Megatron-Bridge)
diff --git a/skills/nemo-mbridge-perf-hierarchical-context-parallel/SKILL.md b/skills/nemo-mbridge-perf-hierarchical-context-parallel/SKILL.md index 68ae9fb5e1..3c2a1ec568 100644 --- a/skills/nemo-mbridge-perf-hierarchical-context-parallel/SKILL.md +++ b/skills/nemo-mbridge-perf-hierarchical-context-parallel/SKILL.md @@ -14,7 +14,7 @@ groups used by `cp_comm_type="a2a+p2p"` and configured with For what hierarchical CP is, when to use it, and the decision tree (`a2a+p2p` vs pure `a2a` vs `p2p`), see: -- @docs/training/hierarchical-context-parallel.md +- @docs/training/hierarchical-context-parallel.mdx - @skills/nemo-mbridge-perf-hierarchical-context-parallel/card.yaml ## Enablement diff --git a/skills/nemo-mbridge-perf-hierarchical-context-parallel/card.yaml b/skills/nemo-mbridge-perf-hierarchical-context-parallel/card.yaml index fb06e916d9..a2288ae94c 100644 --- a/skills/nemo-mbridge-perf-hierarchical-context-parallel/card.yaml +++ b/skills/nemo-mbridge-perf-hierarchical-context-parallel/card.yaml @@ -51,7 +51,7 @@ evidence: - src/megatron/bridge/training/initialize.py - src/megatron/bridge/training/config.py - src/megatron/bridge/training/model_load_save.py - - docs/performance-guide.md + - docs/training/performance-tuning.mdx - tests/unit_tests/training/test_decentralized_pg.py - 3rdparty/Megatron-LM/megatron/core/model_parallel_config.py - 3rdparty/Megatron-LM/megatron/core/parallel_state.py diff --git a/skills/nemo-mbridge-perf-hierarchical-context-parallel/skill-card.md b/skills/nemo-mbridge-perf-hierarchical-context-parallel/skill-card.md index 6b9424cf72..da11f40c7f 100644 --- a/skills/nemo-mbridge-perf-hierarchical-context-parallel/skill-card.md +++ b/skills/nemo-mbridge-perf-hierarchical-context-parallel/skill-card.md @@ -20,7 +20,7 @@ Mitigation: Review and scan skill before deployment.
## Reference(s):
- [Megatron Bridge Documentation](https://docs.nvidia.com/nemo/megatron-bridge/latest/)
-- [Performance Tuning Guide](docs/performance-guide.md)
+- [Performance Tuning Guide](docs/training/performance-tuning.mdx)
- [card.yaml](card.yaml)
diff --git a/skills/nemo-mbridge-perf-megatron-fsdp/SKILL.md b/skills/nemo-mbridge-perf-megatron-fsdp/SKILL.md index 023af6b3cf..58a6dbaa18 100644 --- a/skills/nemo-mbridge-perf-megatron-fsdp/SKILL.md +++ b/skills/nemo-mbridge-perf-megatron-fsdp/SKILL.md @@ -9,7 +9,7 @@ when_to_use: Using FSDP-based data parallelism instead of DDP, or tracing an OOM For stable background and recommendation level, see: -- @docs/training/megatron-fsdp.md +- @docs/training/megatron-fsdp.mdx - @skills/nemo-mbridge-perf-megatron-fsdp/card.yaml ## Enablement diff --git a/skills/nemo-mbridge-perf-megatron-fsdp/skill-card.md b/skills/nemo-mbridge-perf-megatron-fsdp/skill-card.md index 74df405ec0..a47ba7ccb3 100644 --- a/skills/nemo-mbridge-perf-megatron-fsdp/skill-card.md +++ b/skills/nemo-mbridge-perf-megatron-fsdp/skill-card.md @@ -19,7 +19,7 @@ Risk: Review before execution as proposals could introduce incorrect or misleadi Mitigation: Review and scan skill before deployment.
## Reference(s):
-- [Megatron Bridge Performance Tuning Guide](docs/performance-guide.md)
+- [Megatron Bridge Performance Tuning Guide](docs/training/performance-tuning.mdx)
- [Megatron Bridge Documentation](https://docs.nvidia.com/nemo/megatron-bridge/latest/)
diff --git a/skills/nemo-mbridge-perf-memory-tuning/SKILL.md b/skills/nemo-mbridge-perf-memory-tuning/SKILL.md index bac0728cb0..925361c420 100644 --- a/skills/nemo-mbridge-perf-memory-tuning/SKILL.md +++ b/skills/nemo-mbridge-perf-memory-tuning/SKILL.md @@ -7,7 +7,7 @@ when_to_use: GPU OOM errors, reducing peak memory, or tracing an OOM regression # Memory Tuning -Stable docs: @docs/parallelisms.md +Stable docs: @docs/training/parallelisms.mdx Card: @skills/nemo-mbridge-perf-memory-tuning/card.yaml ## What It Is @@ -119,7 +119,7 @@ which eliminated memory fragmentation. The VPP=10 run actually used slightly **more** peak memory (60.2 GB vs 58.8 GB) but did not OOM because expandable segments prevented fragmentation. -VPP should be tuned for pipeline bubble reduction (see @docs/parallelisms.md), +VPP should be tuned for pipeline bubble reduction (see @docs/training/parallelisms.mdx), not as a memory fix. ## Compatibility and Constraints @@ -207,7 +207,7 @@ Selective activation recompute with `mlp` saved ~3 GB peak memory but cost ### Parallelism docs on interleaved pipeline schedule -```116:124:docs/parallelisms.md +```116:124:docs/training/parallelisms.mdx To minimize the pipeline bubble, the computation on each GPU can be divided into multiple subsets of layers (referred to as model chunks), rather than a single contiguous block. Enable this by setting `virtual_pipeline_model_parallel_size`: model_config = GPTModelProvider( diff --git a/skills/nemo-mbridge-perf-memory-tuning/card.yaml b/skills/nemo-mbridge-perf-memory-tuning/card.yaml index f613fd440a..1cf6832fce 100644 --- a/skills/nemo-mbridge-perf-memory-tuning/card.yaml +++ b/skills/nemo-mbridge-perf-memory-tuning/card.yaml @@ -164,8 +164,8 @@ known_limitations: - parallelism resizing (TP/PP) has significant throughput costs - no automatic memory profiling to recommend the optimal strategy evidence: - - docs/parallelisms.md - - docs/performance-guide.md + - docs/training/parallelisms.mdx + - docs/training/performance-tuning.mdx - "PR #3107 (Llama3 70B SFT OOM fix experiment)" - 3rdparty/Megatron-LM/megatron/core/transformer/moe/README.md follow_up_validation: diff --git a/skills/nemo-mbridge-perf-memory-tuning/skill-card.md b/skills/nemo-mbridge-perf-memory-tuning/skill-card.md index 09536cc8f1..1445149ea9 100644 --- a/skills/nemo-mbridge-perf-memory-tuning/skill-card.md +++ b/skills/nemo-mbridge-perf-memory-tuning/skill-card.md @@ -19,8 +19,8 @@ Risk: Review before execution as proposals could introduce incorrect or misleadi Mitigation: Review and scan skill before deployment.
## Reference(s):
-- [Performance Tuning Guide](docs/performance-guide.md)
-- [Parallelism Documentation](docs/parallelisms.md)
+- [Performance Tuning Guide](docs/training/performance-tuning.mdx)
+- [Parallelism Documentation](docs/training/parallelisms.mdx)
- [Activation Recompute Skill](skills/nemo-mbridge-perf-activation-recompute/SKILL.md)
- [Megatron FSDP Skill](skills/nemo-mbridge-perf-megatron-fsdp/SKILL.md)
diff --git a/skills/nemo-mbridge-perf-moe-comm-overlap/SKILL.md b/skills/nemo-mbridge-perf-moe-comm-overlap/SKILL.md index be69bef379..bbd2695829 100644 --- a/skills/nemo-mbridge-perf-moe-comm-overlap/SKILL.md +++ b/skills/nemo-mbridge-perf-moe-comm-overlap/SKILL.md @@ -9,7 +9,7 @@ when_to_use: Tuning MoE communication overlap, or tracing a MoE throughput regre For the higher-level overview, see: -- @docs/training/communication-overlap.md +- @docs/training/communication-overlap.mdx - @skills/nemo-mbridge-perf-moe-comm-overlap/card.yaml ## Quick Decision diff --git a/skills/nemo-mbridge-perf-moe-comm-overlap/card.yaml b/skills/nemo-mbridge-perf-moe-comm-overlap/card.yaml index a70e26ce6f..dc43da2057 100644 --- a/skills/nemo-mbridge-perf-moe-comm-overlap/card.yaml +++ b/skills/nemo-mbridge-perf-moe-comm-overlap/card.yaml @@ -35,8 +35,8 @@ known_limitations: - Public recipes are often conservative and leave MoE overlap disabled by default. - Repo evidence is stronger for validation logic than for end-to-end throughput gains. evidence: - - docs/training/communication-overlap.md - - docs/parallelisms.md + - docs/training/communication-overlap.mdx + - docs/training/parallelisms.mdx - src/megatron/bridge/training/comm_overlap.py - src/megatron/bridge/training/flex_dispatcher_backend.py - src/megatron/bridge/training/config.py diff --git a/skills/nemo-mbridge-perf-moe-comm-overlap/skill-card.md b/skills/nemo-mbridge-perf-moe-comm-overlap/skill-card.md index 18c9d06d28..edcc30b67e 100644 --- a/skills/nemo-mbridge-perf-moe-comm-overlap/skill-card.md +++ b/skills/nemo-mbridge-perf-moe-comm-overlap/skill-card.md @@ -19,10 +19,10 @@ Risk: Review before execution as proposals could introduce incorrect or misleadi Mitigation: Review and scan skill before deployment.
## Reference(s):
-- [Communication Overlap Guide](docs/training/communication-overlap.md)
+- [Communication Overlap Guide](docs/training/communication-overlap.mdx)
- [Comm Overlap Validation Source](src/megatron/bridge/training/comm_overlap.py)
- [Flex Dispatcher Backend Source](src/megatron/bridge/training/flex_dispatcher_backend.py)
-- [Performance Tuning Guide](docs/performance-guide.md)
+- [Performance Tuning Guide](docs/training/performance-tuning.mdx)
## Skill Output:
diff --git a/skills/nemo-mbridge-perf-moe-dispatcher-selection/SKILL.md b/skills/nemo-mbridge-perf-moe-dispatcher-selection/SKILL.md index d7795ac281..3acbe82dbd 100644 --- a/skills/nemo-mbridge-perf-moe-dispatcher-selection/SKILL.md +++ b/skills/nemo-mbridge-perf-moe-dispatcher-selection/SKILL.md @@ -7,7 +7,7 @@ when_to_use: Choosing a MoE token dispatcher, or tracing a MoE regression or cra # MoE Dispatcher Selection Guide -Stable docs: @docs/training/moe-optimization.md +Stable docs: @docs/training/moe-optimization.mdx Card: @skills/nemo-mbridge-perf-moe-dispatcher-selection/card.yaml ## Quick Decision diff --git a/skills/nemo-mbridge-perf-moe-dispatcher-selection/skill-card.md b/skills/nemo-mbridge-perf-moe-dispatcher-selection/skill-card.md index ef902355cb..7ee665bfa0 100644 --- a/skills/nemo-mbridge-perf-moe-dispatcher-selection/skill-card.md +++ b/skills/nemo-mbridge-perf-moe-dispatcher-selection/skill-card.md @@ -20,7 +20,7 @@ Mitigation: Review and scan skill before deployment.
## Reference(s):
- [MoE Dispatcher Selection Skill Definition](skills/nemo-mbridge-perf-moe-dispatcher-selection/SKILL.md)
-- [Performance Tuning Guide](docs/performance-guide.md)
+- [Performance Tuning Guide](docs/training/performance-tuning.mdx)
- [Megatron Bridge Documentation](https://docs.nvidia.com/nemo/megatron-bridge/latest/)
diff --git a/skills/nemo-mbridge-perf-moe-hardware-configs/SKILL.md b/skills/nemo-mbridge-perf-moe-hardware-configs/SKILL.md index 3a588a269c..732e489d9d 100644 --- a/skills/nemo-mbridge-perf-moe-hardware-configs/SKILL.md +++ b/skills/nemo-mbridge-perf-moe-hardware-configs/SKILL.md @@ -7,7 +7,7 @@ when_to_use: Hardware-specific MoE playbooks or throughput estimates; 'MoE on H1 # MoE Hardware Configuration Reference -Stable docs: @docs/training/moe-optimization.md +Stable docs: @docs/training/moe-optimization.mdx Card: @skills/nemo-mbridge-perf-moe-hardware-configs/card.yaml ## Quick Platform Playbook diff --git a/skills/nemo-mbridge-perf-moe-hardware-configs/skill-card.md b/skills/nemo-mbridge-perf-moe-hardware-configs/skill-card.md index f611f0aac5..b23c8c2cea 100644 --- a/skills/nemo-mbridge-perf-moe-hardware-configs/skill-card.md +++ b/skills/nemo-mbridge-perf-moe-hardware-configs/skill-card.md @@ -19,8 +19,8 @@ Risk: Review before execution as proposals could introduce incorrect or misleadi Mitigation: Review and scan skill before deployment.
## Reference(s):
-- [Performance Tuning Guide](docs/performance-guide.md)
-- [Performance Summary Archive](docs/performance-summary-archive.md)
+- [Performance Tuning Guide](docs/training/performance-tuning.mdx)
+- [Performance Summary Archive](docs/reference/performance/archive.mdx)
- [NVIDIA Megatron Bridge Documentation](https://docs.nvidia.com/nemo/megatron-bridge/latest/)
diff --git a/skills/nemo-mbridge-perf-moe-long-context/SKILL.md b/skills/nemo-mbridge-perf-moe-long-context/SKILL.md index abfe8ca952..e7b34e91b5 100644 --- a/skills/nemo-mbridge-perf-moe-long-context/SKILL.md +++ b/skills/nemo-mbridge-perf-moe-long-context/SKILL.md @@ -7,7 +7,7 @@ when_to_use: Training MoE at long sequence lengths, or investigating a commit th # MoE Long-Context Training -Stable docs: @docs/training/moe-optimization.md +Stable docs: @docs/training/moe-optimization.mdx Card: @skills/nemo-mbridge-perf-moe-long-context/card.yaml ## What Changes At Long Context @@ -117,7 +117,7 @@ For long-context MoE training: Useful references: -- @docs/training/activation-recomputation.md +- @docs/training/activation-recomputation.mdx - @skills/nemo-mbridge-perf-cuda-graphs/SKILL.md ## Pitfalls diff --git a/skills/nemo-mbridge-perf-moe-long-context/skill-card.md b/skills/nemo-mbridge-perf-moe-long-context/skill-card.md index c3992aa0b8..75151b3adf 100644 --- a/skills/nemo-mbridge-perf-moe-long-context/skill-card.md +++ b/skills/nemo-mbridge-perf-moe-long-context/skill-card.md @@ -19,9 +19,9 @@ Risk: Review before execution as proposals could introduce incorrect or misleadi Mitigation: Review and scan skill before deployment.
## Reference(s):
-- [MoE Optimization Docs](docs/training/moe-optimization.md)
-- [Activation Recomputation Docs](docs/training/activation-recomputation.md)
-- [Performance Tuning Guide](docs/performance-guide.md)
+- [MoE Optimization Docs](docs/training/moe-optimization.mdx)
+- [Activation Recomputation Docs](docs/training/activation-recomputation.mdx)
+- [Performance Tuning Guide](docs/training/performance-tuning.mdx)
## Skill Output:
diff --git a/skills/nemo-mbridge-perf-moe-optimization-workflow/SKILL.md b/skills/nemo-mbridge-perf-moe-optimization-workflow/SKILL.md index e36a13b77a..91e03e6003 100644 --- a/skills/nemo-mbridge-perf-moe-optimization-workflow/SKILL.md +++ b/skills/nemo-mbridge-perf-moe-optimization-workflow/SKILL.md @@ -7,7 +7,7 @@ when_to_use: Full MoE throughput tuning sweep, or diagnosing a MoE throughput re # MoE Training Optimization Workflow -Stable docs: @docs/training/moe-optimization.md +Stable docs: @docs/training/moe-optimization.mdx Card: @skills/nemo-mbridge-perf-moe-optimization-workflow/card.yaml Source: [Scalable Training of MoE Models with Megatron Core](https://arxiv.org/abs/2603.07685) @@ -156,8 +156,8 @@ or tightly controlled static-shape experiments. Related references: - @skills/nemo-mbridge-perf-cuda-graphs/SKILL.md -- @docs/training/cuda-graphs.md -- @docs/training/activation-recomputation.md +- @docs/training/cuda-graphs.mdx +- @docs/training/activation-recomputation.mdx ## Pitfalls diff --git a/skills/nemo-mbridge-perf-moe-vlm-training/SKILL.md b/skills/nemo-mbridge-perf-moe-vlm-training/SKILL.md index afb1399460..abfd40e2c0 100644 --- a/skills/nemo-mbridge-perf-moe-vlm-training/SKILL.md +++ b/skills/nemo-mbridge-perf-moe-vlm-training/SKILL.md @@ -7,7 +7,7 @@ when_to_use: Training MoE VLMs, or investigating a commit that caused MoE VLM tr # MoE VLM Training -Stable docs: @docs/training/moe-optimization.md +Stable docs: @docs/training/moe-optimization.mdx Card: @skills/nemo-mbridge-perf-moe-vlm-training/card.yaml ## FSDP vs 3D Parallel diff --git a/skills/nemo-mbridge-perf-moe-vlm-training/skill-card.md b/skills/nemo-mbridge-perf-moe-vlm-training/skill-card.md index c53d403154..922c720375 100644 --- a/skills/nemo-mbridge-perf-moe-vlm-training/skill-card.md +++ b/skills/nemo-mbridge-perf-moe-vlm-training/skill-card.md @@ -19,7 +19,7 @@ Risk: Review before execution as proposals could introduce incorrect or misleadi Mitigation: Review and scan skill before deployment.
## Reference(s):
-- [Performance Tuning Guide](docs/performance-guide.md)
+- [Performance Tuning Guide](docs/training/performance-tuning.mdx)
- [Megatron Bridge Documentation](https://docs.nvidia.com/nemo/megatron-bridge/latest/)
diff --git a/skills/nemo-mbridge-perf-parallelism-strategies/SKILL.md b/skills/nemo-mbridge-perf-parallelism-strategies/SKILL.md index 183165067d..eb68b789d5 100644 --- a/skills/nemo-mbridge-perf-parallelism-strategies/SKILL.md +++ b/skills/nemo-mbridge-perf-parallelism-strategies/SKILL.md @@ -9,7 +9,7 @@ when_to_use: Choosing or sizing TP/DP/PP/CP/EP degrees, or tracing an OOM or reg For stable background on each parallelism type, see: -- @docs/parallelisms.md +- @docs/training/parallelisms.mdx - @skills/nemo-mbridge-perf-parallelism-strategies/card.yaml ## Decision by Model Size @@ -217,7 +217,7 @@ total: ~38 GB per GPU Parallelism dimensions set in model provider: -```66:81:docs/parallelisms.md +```66:81:docs/training/parallelisms.mdx model_config = GPTModelProvider( tensor_model_parallel_size=2, # ... other model parameters @@ -226,7 +226,7 @@ model_config = GPTModelProvider( DP size calculation: -```424:436:docs/parallelisms.md +```424:436:docs/training/parallelisms.mdx data_parallel_size = world_size / (tensor_model_parallel_size ร— pipeline_model_parallel_size ร— context_parallel_size) ``` diff --git a/skills/nemo-mbridge-perf-parallelism-strategies/card.yaml b/skills/nemo-mbridge-perf-parallelism-strategies/card.yaml index ea097fcc66..e707e249fe 100644 --- a/skills/nemo-mbridge-perf-parallelism-strategies/card.yaml +++ b/skills/nemo-mbridge-perf-parallelism-strategies/card.yaml @@ -59,10 +59,10 @@ known_limitations: - Not every parallelism combination has the same level of in-repo functional test coverage. - Memory estimates assume standard Adam optimizer and FP16/BF16 parameters. evidence: - - docs/parallelisms.md - - docs/performance-guide.md - - docs/training/communication-overlap.md - - docs/training/hierarchical-context-parallel.md + - docs/training/parallelisms.mdx + - docs/training/performance-tuning.mdx + - docs/training/communication-overlap.mdx + - docs/training/hierarchical-context-parallel.mdx - src/megatron/bridge/training/initialize.py - src/megatron/bridge/training/config.py - src/megatron/bridge/models/common/unimodal.py diff --git a/skills/nemo-mbridge-perf-parallelism-strategies/skill-card.md b/skills/nemo-mbridge-perf-parallelism-strategies/skill-card.md index c70bc0e9f9..d599071f7d 100644 --- a/skills/nemo-mbridge-perf-parallelism-strategies/skill-card.md +++ b/skills/nemo-mbridge-perf-parallelism-strategies/skill-card.md @@ -19,8 +19,8 @@ Risk: Review before execution as proposals could introduce incorrect or misleadi Mitigation: Review and scan skill before deployment.
## Reference(s):
-- [Parallelisms Documentation](docs/parallelisms.md)
-- [Performance Tuning Guide](docs/performance-guide.md)
+- [Parallelisms Documentation](docs/training/parallelisms.mdx)
+- [Performance Tuning Guide](docs/training/performance-tuning.mdx)
- [Pangu Ultra MoE (arXiv:2504.14960)](https://arxiv.org/pdf/2504.14960)
diff --git a/skills/nemo-mbridge-perf-sequence-packing/SKILL.md b/skills/nemo-mbridge-perf-sequence-packing/SKILL.md index 902ceb7f2d..715b1a70ce 100644 --- a/skills/nemo-mbridge-perf-sequence-packing/SKILL.md +++ b/skills/nemo-mbridge-perf-sequence-packing/SKILL.md @@ -9,7 +9,7 @@ when_to_use: Enabling sequence packing or long-context SFT, or investigating a c For stable background and recommendation level, see: -- @docs/training/packed-sequences.md +- @docs/training/packed-sequences.mdx - @skills/nemo-mbridge-perf-sequence-packing/card.yaml ## Enablement diff --git a/skills/nemo-mbridge-perf-sequence-packing/card.yaml b/skills/nemo-mbridge-perf-sequence-packing/card.yaml index 17d6e4756e..b04994d945 100644 --- a/skills/nemo-mbridge-perf-sequence-packing/card.yaml +++ b/skills/nemo-mbridge-perf-sequence-packing/card.yaml @@ -63,11 +63,11 @@ known_limitations: - Qwen3.5-VL disables pack_sequences_in_batch. - The repo does not contain checked-in benchmark results validating long-context throughput claims. evidence: - - docs/training/packed-sequences.md - - docs/performance-guide.md - - docs/training/multi-token-prediction.md - - docs/models/llama/llama3.md - - docs/training/hierarchical-context-parallel.md + - docs/training/packed-sequences.mdx + - docs/training/performance-tuning.mdx + - docs/training/multi-token-prediction.mdx + - docs/models/llama/llama3.mdx + - docs/training/hierarchical-context-parallel.mdx - src/megatron/bridge/data/datasets/packed_sequence.py - src/megatron/bridge/data/datasets/sft.py - src/megatron/bridge/training/utils/packed_seq_utils.py diff --git a/skills/nemo-mbridge-perf-sequence-packing/skill-card.md b/skills/nemo-mbridge-perf-sequence-packing/skill-card.md index c5208ebc4f..37cb04548f 100644 --- a/skills/nemo-mbridge-perf-sequence-packing/skill-card.md +++ b/skills/nemo-mbridge-perf-sequence-packing/skill-card.md @@ -19,8 +19,8 @@ Risk: Review before execution as proposals could introduce incorrect or misleadi Mitigation: Review and scan skill before deployment.
## Reference(s):
-- [Packed Sequences Documentation](docs/training/packed-sequences.md)
-- [Performance Tuning Guide](docs/performance-guide.md)
+- [Packed Sequences Documentation](docs/training/packed-sequences.mdx)
+- [Performance Tuning Guide](docs/training/performance-tuning.mdx)
- [Megatron Bridge Documentation](https://docs.nvidia.com/nemo/megatron-bridge/latest/)
diff --git a/skills/nemo-mbridge-perf-tp-dp-comm-overlap/SKILL.md b/skills/nemo-mbridge-perf-tp-dp-comm-overlap/SKILL.md index e070013780..b00b557870 100644 --- a/skills/nemo-mbridge-perf-tp-dp-comm-overlap/SKILL.md +++ b/skills/nemo-mbridge-perf-tp-dp-comm-overlap/SKILL.md @@ -9,7 +9,7 @@ when_to_use: Enabling TP/DP/PP comm overlap, or tracing a throughput regression For stable background and recommendation level, see: -- @docs/training/communication-overlap.md +- @docs/training/communication-overlap.mdx ## Enablement diff --git a/skills/nemo-mbridge-perf-tp-dp-comm-overlap/card.yaml b/skills/nemo-mbridge-perf-tp-dp-comm-overlap/card.yaml index 473a27b592..ea8d23c2f0 100644 --- a/skills/nemo-mbridge-perf-tp-dp-comm-overlap/card.yaml +++ b/skills/nemo-mbridge-perf-tp-dp-comm-overlap/card.yaml @@ -39,8 +39,8 @@ known_limitations: - Not every public recipe enables overlap even when the feature exists. - Repo docs do not provide benchmark-backed proof for optimal overlap settings. evidence: - - docs/training/communication-overlap.md - - docs/performance-guide.md + - docs/training/communication-overlap.mdx + - docs/training/performance-tuning.mdx - src/megatron/bridge/training/comm_overlap.py - src/megatron/bridge/training/config.py - src/megatron/bridge/training/mixed_precision.py diff --git a/skills/nemo-mbridge-perf-tp-dp-comm-overlap/skill-card.md b/skills/nemo-mbridge-perf-tp-dp-comm-overlap/skill-card.md index 9cb267f86c..b68135b365 100644 --- a/skills/nemo-mbridge-perf-tp-dp-comm-overlap/skill-card.md +++ b/skills/nemo-mbridge-perf-tp-dp-comm-overlap/skill-card.md @@ -19,7 +19,7 @@ Risk: Review before execution as proposals could introduce incorrect or misleadi Mitigation: Review and scan skill before deployment.
## Reference(s):
-- [Megatron Bridge Performance Tuning Guide](docs/performance-guide.md)
+- [Megatron Bridge Performance Tuning Guide](docs/training/performance-tuning.mdx)
- [Megatron Bridge Documentation](https://docs.nvidia.com/nemo/megatron-bridge/latest/)
- [NVIDIA-NeMo/Megatron-Bridge Repository](https://github.com/NVIDIA-NeMo/Megatron-Bridge)
diff --git a/skills/nemo-mbridge-resiliency/SKILL.md b/skills/nemo-mbridge-resiliency/SKILL.md index 643ade7933..24f207651f 100644 --- a/skills/nemo-mbridge-resiliency/SKILL.md +++ b/skills/nemo-mbridge-resiliency/SKILL.md @@ -7,7 +7,7 @@ when_to_use: Enabling resiliency features, or investigating a commit that caused # Resiliency -Stable docs: @docs/training/resiliency.md, @docs/training/checkpointing.md +Stable docs: @docs/training/resiliency.mdx, @docs/training/checkpointing.mdx Card: @skills/nemo-mbridge-resiliency/card.yaml ## Enablement diff --git a/skills/nemo-mbridge-resiliency/card.yaml b/skills/nemo-mbridge-resiliency/card.yaml index ad6be8b779..509e3e283d 100644 --- a/skills/nemo-mbridge-resiliency/card.yaml +++ b/skills/nemo-mbridge-resiliency/card.yaml @@ -94,8 +94,8 @@ known_limitations: - Not all recipes enable resiliency features by default. - Preemption plugin is Slurm-specific. evidence: - - docs/training/resiliency.md - - docs/training/checkpointing.md + - docs/training/resiliency.mdx + - docs/training/checkpointing.mdx - src/megatron/bridge/training/fault_tolerance.py - src/megatron/bridge/training/nvrx_straggler.py - src/megatron/bridge/training/inprocess_restart.py diff --git a/skills/nemo-mbridge-resiliency/skill-card.md b/skills/nemo-mbridge-resiliency/skill-card.md index bf3dd4418b..116e4da179 100644 --- a/skills/nemo-mbridge-resiliency/skill-card.md +++ b/skills/nemo-mbridge-resiliency/skill-card.md @@ -19,8 +19,8 @@ Risk: Review before execution as proposals could introduce incorrect or misleadi Mitigation: Review and scan skill before deployment.
## Reference(s):
-- [Megatron Bridge Resiliency Documentation](docs/training/resiliency.md)
-- [Megatron Bridge Checkpointing Documentation](docs/training/checkpointing.md)
+- [Megatron Bridge Resiliency Documentation](docs/training/resiliency.mdx)
+- [Megatron Bridge Checkpointing Documentation](docs/training/checkpointing.mdx)
- [Megatron Bridge GitHub Repository](https://github.com/NVIDIA-NeMo/Megatron-Bridge)
From a4f3f28b52e30f00d8bc19734fb690cd1231d8ae Mon Sep 17 00:00:00 2001 From: Andrew Chen Date: Mon, 8 Jun 2026 12:48:30 +0000 Subject: [PATCH 2/5] docs(fern): add v0.4 versioned yml, register in docs.yml, add conformance report MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add docs/fern/versions/v0.4.yml mirroring nightly.yml navigation for the v0.4.x / NeMo Framework 26.04 stable release (latest git tag: v0.4.2). Version source: git tags (v0.4.2 is latest stable) + package_info.py MAJOR=0 MINOR=4 at time of migration. Display name "v0.4 (26.04)" aligns the Megatron-Bridge semver with the NeMo Framework container release label used in software-versions.mdx. - Register the v0.4 version in docs/fern/docs.yml alongside Nightly. - Add docs/fern/CONFORMANCE-REPORT.md: tpl-lint sweep of all 124 MDX pages. Findings: 0 true errors; 37 uniform false-positives (duplicate_h1 from code-block comment lines โ€” known tpl-lint limitation); warnings are orphan_heading style advisories only. Signed-off-by: Andrew Chen --- docs/fern/CONFORMANCE-REPORT.md | 152 ++++++++++++++++++++++++++++++++ docs/fern/docs.yml | 3 + docs/fern/versions/v0.4.yml | 50 +++++++++++ 3 files changed, 205 insertions(+) create mode 100644 docs/fern/CONFORMANCE-REPORT.md create mode 100644 docs/fern/versions/v0.4.yml diff --git a/docs/fern/CONFORMANCE-REPORT.md b/docs/fern/CONFORMANCE-REPORT.md new file mode 100644 index 0000000000..2a02e304cd --- /dev/null +++ b/docs/fern/CONFORMANCE-REPORT.md @@ -0,0 +1,152 @@ +# Fern MDX Migration โ€” Template Conformance Report + +**Date:** 2026-06-08 +**Branch:** docs/fern-mdx-migration +**Linter:** `~/repos/template-library/skills/tpl-lint/scripts/validate.py --auto-detect` +**Pages scanned:** 124 MDX files (all `docs/**/*.mdx` excluding `docs/fern/`) + +--- + +## Summary + +| Section | Pages | Errors | Warnings | Notes | +|---------|------:|-------:|---------:|-------| +| about (root) | 2 | 1 | 8 | 1 false-positive (see ยงKnown issue) | +| about/release-notes | 3 | 0 | 9 | Clean | +| get-started | 7 | 3 | 90 | 2 false-positives + 1 (see below) | +| index (root) | 1 | 0 | 0 | Clean | +| model-optimization | 4 | 0 | 14 | Clean | +| models | 55 | 17 | 299 | 17 false-positives | +| reference | 5 | 0 | 53 | Clean | +| resources | 7 | 0 | 42 | Clean | +| training | 24 | 14 | 120 | 14 false-positives | +| troubleshooting | 2 | 0 | 6 | Clean | +| use-cases | 5 | 2 | 11 | 2 false-positives | +| **TOTAL** | **124** | **37** | **652** | | + +**True errors: 0.** All 37 reported `errors` are a single class of false-positive โ€” see ยงKnown issue below. + +--- + +## Per-Section Details + +### about/ (2 pages) +| File | Schema | Valid | Errors | Warnings | +|------|--------|-------|--------|---------| +| about/architecture.mdx | concept | False | 1 | 4 | +| about/index.mdx | concept | True | 0 | 4 | + +- `architecture.mdx`: 1 `duplicate_h1` false-positive (code block comments). + +### about/release-notes/ (3 pages) โ€” CLEAN +| File | Schema | Valid | Errors | Warnings | +|------|--------|-------|--------|---------| +| about/release-notes/changelog.mdx | procedural | True | 0 | 0 | +| about/release-notes/index.mdx | concept | True | 0 | 4 | +| about/release-notes/software-versions.mdx | concept | True | 0 | 5 | + +### get-started/ (7 pages) +| File | Schema | Valid | Errors | Warnings | +|------|--------|-------|--------|---------| +| get-started/index.mdx | concept | True | 0 | 2 | +| get-started/installation.mdx | concept | True | 0 | 2 | +| get-started/migration/index.mdx | procedural | True | 0 | 0 | +| get-started/migration/megatron-lm-to-megatron-bridge.mdx | concept | False | 1 | 18 | +| get-started/migration/nemo2-to-megatron-bridge.mdx | concept | False | 1 | 58 | +| get-started/prerequisites.mdx | prerequisites | True | 0 | 4 | +| get-started/quickstart/hugging-face-conversion.mdx | concept | False | 1 | 6 | +| get-started/quickstart/index.mdx | concept | True | 0 | 2 | + +- 3 `duplicate_h1` false-positives (code block comments in migration and quickstart pages). + +### model-optimization/ (4 pages) โ€” CLEAN +| File | Schema | Valid | Errors | Warnings | +|------|--------|-------|--------|---------| +| model-optimization/distillation.mdx | concept | True | 0 | 6 | +| model-optimization/index.mdx | concept | True | 0 | 2 | +| model-optimization/pruning.mdx | concept | True | 0 | 1 | +| model-optimization/quantization.mdx | concept | True | 0 | 5 | + +### models/ (55 pages) +Pages with `errors=1` (all false-positive `duplicate_h1`): +`deepseek-v2`, `deepseek-v3`, `gemma2`, `gemma3`, `glm45`, `llama3`, `mistral`, `moonlight`, `nemotronh`, `nemotron3-super`, `llama-nemotron`, `olmoe`, `qwen`, `megatron-lm-to-megatron-bridge`, `add-new-model`, `rl-framework-integration` + +All model index pages (folder `index.mdx` files) are clean (`valid=True, errors=0`). + +Pages with `errors=0` (38 of 55 models pages): all clean. + +### reference/ (5 pages) โ€” CLEAN +| File | Schema | Valid | Errors | Warnings | +|------|--------|-------|--------|---------| +| reference/api/index.mdx | concept | True | 0 | 0 | +| reference/index.mdx | procedural | True | 0 | 0 | +| reference/performance/archive.mdx | concept | True | 0 | 41 | +| reference/performance/index.mdx | procedural | True | 0 | 0 | +| reference/performance/summary.mdx | concept | True | 0 | 12 | + +- `reference/performance/archive.mdx`: 41 warnings (orphan headings โ€” large table-heavy reference page; expected pattern). + +### resources/ (7 pages) โ€” CLEAN +All `valid=True, errors=0`. `documentation-map.mdx` has 31 warnings (all `orphan_heading` from table-only sections โ€” expected for a navigation index page). + +### training/ (24 pages) +| Clean (errors=0) | With false-positive errors | +|------------------|-----------------------------| +| communication-overlap, cpu-offloading, cuda-graphs, entry-points, hierarchical-context-parallel, index, megatron-fsdp, memory-estimator, moe-optimization, multi-token-prediction (wait โ€” see below), optimizer-scheduler, packed-sequences, resiliency, training-loop-settings | activation-recomputation, attention-optimizations, callbacks, checkpointing, config-container-overview, logging, mixed-precision, multi-token-prediction, parallelisms, peft, performance-tuning, profiling, recipe-usage | + +14 pages have `duplicate_h1` errors (all false-positives from Python/YAML comments in code fences). + +### troubleshooting/ (2 pages) โ€” CLEAN +| File | Schema | Valid | Errors | Warnings | +|------|--------|-------|--------|---------| +| troubleshooting/index.mdx | troubleshooting | True | 0 | 2 | +| troubleshooting/known-issues.mdx | concept | True | 0 | 4 | + +### use-cases/ (5 pages) +| File | Schema | Valid | Errors | Warnings | +|------|--------|-------|--------|---------| +| use-cases/index.mdx | procedural | True | 0 | 0 | +| use-cases/model-development/add-new-model.mdx | concept | False | 1 | 7 | +| use-cases/model-development/index.mdx | procedural | True | 0 | 0 | +| use-cases/reinforcement-learning/index.mdx | procedural | True | 0 | 0 | +| use-cases/reinforcement-learning/rl-framework-integration.mdx | concept | False | 1 | 4 | + +- 2 false-positive `duplicate_h1` errors. + +--- + +## Known Issue: `duplicate_h1` False-Positive + +**All 37 reported `error`-level findings are the same rule: `duplicate_h1`.** + +The tpl-lint tool does not distinguish between actual H1 headings (`# Title`) and `#` comment characters inside Python, YAML, or bash code fences. Pages with code examples (especially model-specific pages and training configuration pages) contain many lines like: + +```python +# Example: configure parallelism +# Load the bridge from HF model ID +``` + +These are flagged as H1 headings. This is a known limitation of the tpl-lint `duplicate_h1` rule (no code-fence awareness). + +**Impact:** Zero actual content errors. All pages have exactly one real H1 (`# Page Title`). The false-positive count per page scales with the number of code block `#` comment lines. + +**Recommendation:** File a tpl-lint issue to add code-fence exclusion to the `duplicate_h1` rule. Until fixed, the `valid: false` results for these pages can be ignored for migration-quality assessment purposes. + +--- + +## Warnings Summary + +All warnings are `orphan_heading` (a heading followed immediately by a table, list, or MDX component with no intervening prose paragraph). This is a style advisory, not an error. Pages where this is pervasive: + +- `reference/performance/archive.mdx` (41 warnings) โ€” large reference table; expected. +- `resources/documentation-map.mdx` (31 warnings) โ€” navigation map; expected. +- `training/index.mdx` (12 warnings) โ€” section overview with many headingโ†’table blocks. +- `get-started/migration/nemo2-to-megatron-bridge.mdx` (58 warnings) โ€” content-dense migration guide with many subsection tables. + +These are structural characteristics of the content, not migration defects. + +--- + +## Conclusion + +**Migration quality: CLEAN.** 124 MDX pages pass template conformance with 0 true errors. The 37 reported errors are a uniform tpl-lint false-positive class (code-block `#` comments misidentified as H1 headings). Warnings are all `orphan_heading` style advisories consistent with the content type. The release-related pages (releases section, troubleshooting, resources/contributing) are fully clean. diff --git a/docs/fern/docs.yml b/docs/fern/docs.yml index d336d74771..c26849331c 100644 --- a/docs/fern/docs.yml +++ b/docs/fern/docs.yml @@ -19,6 +19,9 @@ versions: - display-name: Nightly path: versions/nightly.yml slug: nightly + - display-name: "v0.4 (26.04)" + path: versions/v0.4.yml + slug: v0.4 redirects: - source: "/nemo/megatron-bridge/index.html" diff --git a/docs/fern/versions/v0.4.yml b/docs/fern/versions/v0.4.yml new file mode 100644 index 0000000000..fd8f7c527f --- /dev/null +++ b/docs/fern/versions/v0.4.yml @@ -0,0 +1,50 @@ +navigation: + - section: "Documentation" + contents: + - page: "Home" + path: ../../index.mdx + - section: "About" + contents: + - folder: ../../about + title: "Overview" + title-source: frontmatter + - section: "Get Started" + contents: + - folder: ../../get-started + title: "Getting Started" + title-source: frontmatter + - section: "Supported Models" + contents: + - folder: ../../models + title: "Supported Models" + title-source: frontmatter + - section: "Training and Customization" + contents: + - folder: ../../training + title: "Training and Customization" + title-source: frontmatter + - section: "Model Optimization" + contents: + - folder: ../../model-optimization + title: "Model Optimization" + title-source: frontmatter + - section: "Use Cases" + contents: + - folder: ../../use-cases + title: "Use Cases" + title-source: frontmatter + - section: "Reference" + contents: + - folder: ../../reference + title: "Reference" + title-source: frontmatter + - section: "Troubleshooting" + contents: + - folder: ../../troubleshooting + title: "Troubleshooting" + title-source: frontmatter + - section: "Resources" + contents: + - folder: ../../resources + title: "Resources" + title-source: frontmatter From 3db9eb30c91f6b8fb9a8f4d60a9fcd6aa89d0102 Mon Sep 17 00:00:00 2001 From: Andrew Chen Date: Mon, 8 Jun 2026 06:41:51 -0700 Subject: [PATCH 3/5] docs(fern): fix broken image refs in parallelisms.mdx The page was relocated to docs/training/ but kept images/X relative refs, which resolved to docs/training/images/ (unrelated assets) instead of the shared docs/images/. Repath the 6 refs to ../images/X, matching sibling training pages' convention for shared images. fern check now reports 0 errors. Signed-off-by: Andrew Chen --- docs/training/parallelisms.mdx | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/training/parallelisms.mdx b/docs/training/parallelisms.mdx index 83e7af6963..fd5998311d 100644 --- a/docs/training/parallelisms.mdx +++ b/docs/training/parallelisms.mdx @@ -15,7 +15,7 @@ Data Parallelism (DP) replicates the model across multiple GPUs. Data batches ar Distributed Data Parallelism (DDP) keeps the model copies consistent by synchronizing parameter gradients across data-parallel GPUs before each parameter update. More specifically, it sums the gradients of all model copies using all-reduce communication collectives. -![Distributed Data Parallelism](images/ddp.gif) +![Distributed Data Parallelism](../images/ddp.gif) *Figure: Distributed Data Parallelism synchronizes gradients across multiple GPUs using all-reduce operations.* ### Distributed Optimizer @@ -59,10 +59,10 @@ Model Parallelism (MP) is a distributed model deployment method that partitions Tensor Parallelism (TP) is a model-parallel partitioning method that distributes the parameter tensor of an individual layer across GPUs. In addition to reducing model state memory usage, it also saves activation memory as the per-GPU tensor sizes shrink. However, the reduced per-GPU tensor size increases CPU overhead due to smaller per-GPU kernel workloads. -![Tensor Parallelism Overview](images/tp1.png) +![Tensor Parallelism Overview](../images/tp1.png) *Figure 1: Tensor Parallelism distributes individual layer parameters across multiple GPUs.* -![Tensor Parallelism Implementation](images/tp2.png) +![Tensor Parallelism Implementation](../images/tp2.png) *Figure 2: Detailed view of how tensor parallelism splits weight matrices and synchronizes computations.* #### Enable Tensor Parallelism @@ -93,7 +93,7 @@ Megatron Bridge integrates TP through the implementation from Megatron Core. For Pipeline Parallelism (PP) is a technique that assigns consecutive layers or segments of a neural network to different GPUs. This division allows each GPU to process different stages of the network sequentially. -![Pipeline Parallelism](images/pp.gif) +![Pipeline Parallelism](../images/pp.gif) *Figure: Pipeline Parallelism distributes consecutive layers across multiple GPUs, processing batches in a pipeline fashion.* #### Enable Pipeline Parallelism @@ -140,7 +140,7 @@ Expert Parallelism (EP) is a type of model parallelism that distributes experts MoE is a machine learning technique where multiple specialized models (experts, usually multi-layer perceptrons) are combined to solve a complex task. Each expert focuses on a specific subtask or domain, while a gating network dynamically activates the most appropriate expert based on the current input. -![Expert Parallelism](images/ep.png) +![Expert Parallelism](../images/ep.png) *Figure: Expert Parallelism distributes MoE experts across multiple GPUs while keeping other layers replicated.* #### Basic MoE Configuration @@ -333,7 +333,7 @@ In LLM training, a large memory space is needed to store the input activations o Sequence Parallelism (SP) extends tensor-level model parallelism by distributing computing load and activation memory across multiple GPUs along the sequence dimension of transformer layers. This method is particularly useful for portions of the layer that have previously not been parallelized, enhancing overall model performance and efficiency. -![Sequence Parallelism](images/sp.png) +![Sequence Parallelism](../images/sp.png) *Figure: Sequence Parallelism distributes the sequence dimension across multiple GPUs, reducing activation memory.* #### Enable Sequence Parallelism From 86b790d7833150c39fe1570b868415dd0fa72ef2 Mon Sep 17 00:00:00 2001 From: Andrew Chen Date: Mon, 8 Jun 2026 12:34:37 -0700 Subject: [PATCH 4/5] docs(fern): fix MDX syntax errors blocking the build fern docs dev failed to parse 3 files (defects fern check missed): - nemo2-to-megatron-bridge.mdx + training/performance-tuning.mdx: un-converted HTML comments () -> MDX comment syntax ({/* */}). - training/hierarchical-context-parallel.mdx: raw <= read as a JSX tag -> escaped (\<=). Verified: all 124 MDX compile clean via @mdx-js/mdx batch compile. Signed-off-by: Andrew Chen --- .../migration/nemo2-to-megatron-bridge.mdx | 2 +- .../hierarchical-context-parallel.mdx | 4 +- docs/training/performance-tuning.mdx | 42 +++++++++---------- docs/training/recipe-usage.mdx | 2 +- 4 files changed, 25 insertions(+), 25 deletions(-) diff --git a/docs/get-started/migration/nemo2-to-megatron-bridge.mdx b/docs/get-started/migration/nemo2-to-megatron-bridge.mdx index 670b5e8533..0cd4c130c2 100644 --- a/docs/get-started/migration/nemo2-to-megatron-bridge.mdx +++ b/docs/get-started/migration/nemo2-to-megatron-bridge.mdx @@ -41,7 +41,7 @@ Megatron Bridge supports the following model families through bridge mappings an For a complete list of bridge mappings, providers, and their parameters, see `bridge.models`. - +{/* TODO: Create a dedicated model support table with tested HF checkpoint mappings */} --- diff --git a/docs/training/hierarchical-context-parallel.mdx b/docs/training/hierarchical-context-parallel.mdx index 7f98609d92..f4f114d94b 100644 --- a/docs/training/hierarchical-context-parallel.mdx +++ b/docs/training/hierarchical-context-parallel.mdx @@ -22,7 +22,7 @@ several CP communication backends: |---|---|---|---| | `"p2p"` | Ring-exchange of KV chunks | Yes | None | | `"all_gather"` | All-gather full KV before attention | No | None | -| `"a2a"` | All-to-all: scatter heads, gather full sequence (Ulysses-style) | N/A | **CP <= num_kv_heads** | +| `"a2a"` | All-to-all: scatter heads, gather full sequence (Ulysses-style) | N/A | **CP \<= num_kv_heads** | | `"a2a+p2p"` | Hierarchical: a2a within inner group, p2p across outer group | Partial (p2p part) | Requires `hierarchical_context_parallel_sizes` | **HCP (`a2a+p2p`)** exists to scale CP beyond the KV head count by combining @@ -70,7 +70,7 @@ Model: 8 KV heads. Cluster: 4 nodes x 8 GPUs. Goal: train 128K sequences. | Option | TP | CP | `cp_comm_type` | Notes | |---|---|---|---|---| | A | 1 | 16 | `a2a+p2p` with `[8,2]` | a2a intra-node (8 GPUs), p2p across 2 node-groups | -| B | 2 | 4 | `a2a` | CP=4 <= 8 KV heads. Simpler. Often same throughput. | +| B | 2 | 4 | `a2a` | CP=4 \<= 8 KV heads. Simpler. Often same throughput. | | C | 1 | 16 | `p2p` | Works but no a2a bandwidth benefit intra-node | In practice, **option B is usually preferred** -- benchmarks showed identical diff --git a/docs/training/performance-tuning.mdx b/docs/training/performance-tuning.mdx index e36b00d8be..c288a9fae5 100644 --- a/docs/training/performance-tuning.mdx +++ b/docs/training/performance-tuning.mdx @@ -110,23 +110,23 @@ Additionally, because CP shards activations, it also partitions optimizer states > > 2. In FSDP training, activation storage remains as the main memory bottleneck because FSDP only shards model state memory, and a large per-GPU activation is needed to hide the costly FSDP communication. On GB200 GPUs, Megatron-Bridge offers an option to offload activations to the host memory via a high-speed chip-to-chip interconnect. > > 3. Baseline training is host performance-bound, but FSDP allows for larger per-GPU tensor sizes by eliminating TP or enabling a larger micro-batch size. - - - - - - - - - - - - - - - - - + {/* TODO: support megatron custom fsdp */} + {/* > 1. Megatron-Bridge supports two Fully Sharded Data Parallelism (FSDP) implementations: PyTorch-native FSDP and a custom Megatron FSDP built within Megatron Core. While both follow the same sharding principles, the custom implementation is further optimized for performance. The performance gain of the custom FSDP comes primarily from minimizing the data movement to the communication tensors and reusing communication buffers. Both FSDP methods can be used in combination with per-tensor sharding methods. */} + {/* > */} + {/* > > 1. To use PyTorch FSDP2: */} + {/* > > */} + {/* > > > 1. `DistributedInitConfig.use_torch_fsdp2=True` */} + {/* > > */} + {/* > > 2. To use Custom Megatron FSDP: */} + {/* > > */} + {/* > > > 1. `recipe.trainer.strategy.fsdp="megatron"` */} + {/* > > > 2. `recipe.trainer.strategy.ddp.data_parallel_sharding_strategy="optim_grads_params"` */} + {/* > */} + {/* > 2. FSDP can be preferred over TP+PP+DP mappings in the following scenarios: */} + {/* > */} + {/* > > 1. Small models with a large sequence, thus the parameter AllGather and gradient ReduceScatter can effectively be hidden under computation and the short communication overlap causes minor interference to the computation under overlap. */} + {/* > > 2. In FSDP training, activation storage remains as the main memory bottleneck because FSDP only shards model state memory, and a large per-GPU activation is needed to hide the costly FSDP communication. On GB200 GPUs, Megatron-Bridge offers an option to offload activations to the host memory via a high-speed chip-to-chip interconnect. */} + {/* > > 3. Baseline training is host performance-bound, but FSDP allows for larger per-GPU tensor sizes by eliminating TP or enabling a larger micro-batch size. */} 4. Heterogeneous Encoder Parallelism @@ -188,9 +188,9 @@ Additionally, because CP shards activations, it also partitions optimizer states > > > 1. `NVTE_FWD_LAYERNORM_SM_MARGIN=<#SM for DP collectives = 16>` > > > 2. `NVTE_BWD_LAYERNORM_SM_MARGIN=<#SM for DP collectives = 16>` - +{/* 2. Custom Megatron FSDP */} - +{/* > 1. Unless you specify the communication bucket size, MCORE FSDP uses fixed communication overlap that overlaps the parameter AllGather and gradient ReduceScatter of each Transformer layer with its associated forward and backward computations. */} 3. Tensor-parallel (TP) communication (with sequence parallelism) @@ -793,5 +793,5 @@ uv run python -u /home/dpsk_a2a/deepep/tests/test_internode.py - `TPOverlapCfg.fp8_buf` - `TPOverlapCfg.num_sm` - `TPOverlapCfg.num_split` - - +{/* - `garbageCollectionCallback.gc_interval_val` */} +{/* - `NsysPlugin` */} diff --git a/docs/training/recipe-usage.mdx b/docs/training/recipe-usage.mdx index d38d81d047..427a5b67b5 100644 --- a/docs/training/recipe-usage.mdx +++ b/docs/training/recipe-usage.mdx @@ -156,7 +156,7 @@ finetune_cfg.dataset.dataset_root = "/data/sft_jsonl" finetune_cfg.dataset.seq_length = 4096 ``` -For more detail on accepted dataset layouts, see [Data Preparation](training/data-preparation.md). +For more detail on accepted dataset layouts, see [Data Preparation](/training/data-preparation). ## Launch methods From a36c7c4df50e69c88ae9e8bf78907fee4c422b80 Mon Sep 17 00:00:00 2001 From: Andrew Chen Date: Tue, 16 Jun 2026 08:39:41 -0700 Subject: [PATCH 5/5] docs(fern): convert leftover {py:class} MyST role to a code span MDX parses {...} as a JS expression, so the unconverted Sphinx/MyST role {py:class}`...` broke the Fern docs build (acorn parse error on training/mixed-precision.mdx). Drop the role wrapper, keep the code span. This was the only remaining {role}`...` / :role:`...` instance in the migration (verified by scan); the preview now builds (126 pages). Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/training/mixed-precision.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/training/mixed-precision.mdx b/docs/training/mixed-precision.mdx index 3a51b691f3..ce5bae1373 100644 --- a/docs/training/mixed-precision.mdx +++ b/docs/training/mixed-precision.mdx @@ -98,7 +98,7 @@ config = ConfigContainer( ## NVFP4 Training -NVFP4 is a 4-bit floating-point training mode for Blackwell-generation GPUs, implemented through Transformer Engine. In Megatron Bridge it is configured through the `fp4` fields on {py:class}`bridge.training.mixed_precision.MixedPrecisionConfig`, not through the FP8 fields. +NVFP4 is a 4-bit floating-point training mode for Blackwell-generation GPUs, implemented through Transformer Engine. In Megatron Bridge it is configured through the `fp4` fields on `bridge.training.mixed_precision.MixedPrecisionConfig`, not through the FP8 fields. NVFP4 and FP8 are mutually exclusive. If `fp4` and `fp8` are both set, configuration finalization raises an error. NVFP4 also requires a Transformer Engine build with NVFP4 block-scaling support.