Skip to content

fix(peft): preserve layer boundaries in wildcard selectors - #6407

Draft
yaoyu-33 wants to merge 1 commit into
mainfrom
landscape/peft-wildcard-separators-20261003
Draft

yaoyu-33 wants to merge 1 commit into
mainfrom
landscape/peft-wildcard-separators-20261003

Conversation

@yaoyu-33

Copy link
Copy Markdown
Contributor

The documented PEFT selectors for layers 0 and 1 also match layers 10–19 because wildcard_match treats a literal path separator (.) as a regex wildcard. This can silently attach adapters to extra layers, or exclude those layers unexpectedly.

Escape periods before expanding *. A regression test exercises the existing ModuleMatcher target, exclusion and canonical-mapping paths over 24 layers. The production change is one line.

Evidence: the documented layer selectors and matcher helper. This follows the selector-contract investigation prompted by Automodel #4045; no implementation was copied. Related issue #1738 concerns unmatched targets, while this fixes overmatching. The candidate was carried forward from the weekly landscape report.

Validation on base 2161fe922dc97e503eb1d52d644483066d8e99b0, using the pinned MCore dependency and CPU tensors:

  • Fail before / pass after: uv run python -m pytest tests/unit_tests/peft/test_module_matcher.py::TestModuleMatcher::test_layer_wildcards_preserve_path_separators -q — 3 assertion failures before the production change; 3 passed afterward. The initial target/canonical selection included layers 10–19, and the exclusion path incorrectly omitted them.
  • Adjacent tests: uv run python -m pytest tests/unit_tests/peft/test_module_matcher.py tests/unit_tests/peft/test_utils.py -k 'TestModuleMatcher or wildcard_match' -q — 37 passed, 119 deselected.
  • uv run pre-commit run --all-files and git diff --check passed.
  • uv run mypy --strict --no-incremental src/megatron/bridge/peft/utils.py tests/unit_tests/peft/test_module_matcher.py reports 139 existing diagnostics on both the base and patched trees, with identical diagnostics after accounting for line offsets.
  • Separate read-only checklist review completed after the reviewer service could not start. Duplicate, ownership, minimal scope, red/green and publication checks passed.

Current main was rechecked at aea5c536903456835dabdeee7cb0925ba8208bdc; the intervening PP-local export change does not touch the PEFT source, tests or documentation. No matching issue/PR implementation was found.

Non-goals: expanding the matcher dialect, changing adapter/model scope, adding dependencies, modifying CI or public conversion signatures, or validating distributed training. This Draft fixes the documented layer-selection contract only.

Signed-off-by: Yu Yao <yaoyu.094@gmail.com>
@copy-pr-bot

copy-pr-bot Bot commented Oct 10, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@yaoyu-33

Copy link
Copy Markdown
Contributor Author

/ok to test 4da9918

@github-actions

Copy link
Copy Markdown
Contributor

@cuichenx cuichenx added bug Something isn't working area:peft Parameter-efficient fine-tuning (LoRA, adapters) labels Oct 10, 2026

This branch was successfully deployed

1 active deployment
test — 4da99181 Deployed Oct 10, 2026 by copy-pr-bot[bot] via cicd-wait-in-queue #22743
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:peft Parameter-efficient fine-tuning (LoRA, adapters) bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants