Skip to content

build: AUT-2524 bump MCore main to a98aed55 - #6438

Merged
ko3n1g merged 5 commits into
NVIDIA-NeMo:mainfrom
svcnemo-autobot:chore/bump-mcore-main-261012-a98aed55
Oct 13, 2026
Merged

ko3n1g merged 5 commits into
NVIDIA-NeMo:mainfrom
svcnemo-autobot:chore/bump-mcore-main-261012-a98aed55

Conversation

@svcnemo-autobot

@svcnemo-autobot svcnemo-autobot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Background

Nightly Megatron-Core bump for the main lane, moving Bridge's MCore pin to a98aed55. Bridge main has since moved its own pin to 8c815a4c, an ancestor of a98aed55, and upstream main was merged in at 1edea5bc.

What changed

  • .main.commit and the 3rdparty/Megatron-LM gitlink now point to a98aed553e25b4e86c59f50be54ac0ea4227b442. .dev.commit is unchanged.
  • uv.lock matches main. uv lock --check passes with uv 0.7.2, the docker/Dockerfile.ci pin. MCore's dependency metadata is unchanged at a98aed55.
  • L0_Launch_models_bert.sh takes its coverage --data-file/--source paths from the repo root instead of a hardcoded /opt/Megatron-Bridge.

Details

Tests were selected by tracing the MCore changes since e30d863a: distributed-optimizer checkpoint layout, deterministic GDN l2norm, empty hybrid PP stages, and inference stop/sampling. That gave 77 rows: 4 unit rows (Core in 3 shards, plus Diffusion) and 73 H100 functional launchers. After the main merge, 7 rows were re-run to cover its CUDA-graph validation/teardown and nemotronh perf-recipe changes.

Tested

Run locally on H100 with the CI FlashAttention env (NVTE_FLASH_ATTN_V2=1, V3/V4 off).

  • On merged head 52c1a6de with image sha256:68b61252df0c1a0eb6277494c99c2031fd0d042f08d82ae9864d8f7cef8d7e20, 7 rows passed with 0 failures: Core unit shards 0-2 (4045, 4017 and 4043 tests), L0_Launch_megatron_fsdp, L0_Launch_training_pretrain, L0_Launch_recipes_perf_config and L2_Launch_recipes_llama_cuda_graphs.
  • The other 68 selected rows passed on the pre-merge head with image sha256:65858877. Their results still apply: the MCore pin, Dockerfile and pyproject are unchanged, and no runtime code imports the packages bumped in the lock.
  • 2 rows were not run, by requester decision, because their test data is missing: L0_Launch_recipes_deepseek.sh and L1_Launch_recipes_glm_vl.sh. Upstream CI skips them too.
  • GitHub CI on this head: pending.

Update .main.commit and the 3rdparty/Megatron-LM gitlink from
e0c5e18673bf4d4fd89cf87c4f1afbe521fe297f to
e30d863a6d82a3c45f2d71355dc73fbf5610967b, align the TransformerEngine
dependency metadata version with MCore (2.20.2+44f31bbe), and refresh
uv.lock with uv lock --upgrade.

Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
Requester-authorized pin revision: update .main.commit and the
3rdparty/Megatron-LM gitlink from e30d863a6d82a3c45f2d71355dc73fbf5610967b
to a98aed553e25b4e86c59f50be54ac0ea4227b442 (NVIDIA/Megatron-LM main).
MCore dependency metadata is unchanged; uv lock --check passes.
.dev.commit unchanged.

Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
…-261012-main

Upstream main landed NVIDIA-NeMo#6414 (MCore main e30d863a). Resolve the
.main.commit conflict to the session pin a98aed553e25b4e86c59f50be54ac0ea4227b442;
the 3rdparty/Megatron-LM gitlink fast-forwards to the same commit.
Keep the session uv lock --upgrade output (pyproject.toml is identical on
both sides); uv lock --check passes on the merged tree.

Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
L0_Launch_models_bert.sh hardcoded /opt/Megatron-Bridge for the
coverage data file and source root. When the suite runs from a
checkout other than /opt/Megatron-Bridge (or that path is read-only),
coverage cannot write its data file and the row fails before pytest
results are reported. Resolve REPO_ROOT from the script location and
use it for --data-file and --source.

Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Oct 13, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@svcnvidia-nemo-ci svcnvidia-nemo-ci added area:build Dependencies, packaging, images, and environment setup full-test-suite labels Oct 13, 2026
@svcnvidia-nemo-ci

Copy link
Copy Markdown
Contributor

/ok to test 33db331

@github-actions

Copy link
Copy Markdown
Contributor

Automatic Claude reviews have been retired. To request a pull-request review, post a comment containing:

/review

Add model=claude to use a Claude reviewer (the default is model=codex). mode=light|strict selects the review depth; for example, /review model=claude mode=strict. Comment /review help for all options.

…main-261012-a98aed55

Keep the MCore main pin a98aed55; main's 8c815a4c is its ancestor.
Take main's uv.lock; uv lock --check passes for the merged tree.

Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
@svcnvidia-nemo-ci

Copy link
Copy Markdown
Contributor

/ok to test 52c1a6d

@ko3n1g
ko3n1g enabled auto-merge (squash) October 13, 2026 18:44
@ko3n1g
ko3n1g merged commit bbd37ba into NVIDIA-NeMo:main Oct 13, 2026
168 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:build Dependencies, packaging, images, and environment setup full-test-suite

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants