[model] fix: clamp the first CP chunk slice in preprocess_packed_seqs

[gaohongkui]

Problem

In preprocess_packed_seqs, when cp_size > 1, the chunk indices are computed in the
alignment-padded coordinate space (seqlen_padded_i, aligned to tp_size * cp_size * 2),
while d = input_ids[i, attention_mask[i]] holds only the valid tokens.

When a row's valid length is shorter than the alignment padding, the first-chunk slice
d[half_seqlen * cp_rank : half_seqlen * (cp_rank + 1)] can start past the end of d and
yield an empty tensor, so the assignment raises:

RuntimeError: The expanded size of the tensor (2) must match the existing size (0)
at non-singleton dimension 0.  Target sizes: [2].  Tensor sizes: [0]

The remaining chunk immediately below is already clamped with
min(remain_end, d.shape[0]) — only the first chunk was missed. The identical code in verl
was fixed in verl-project/verl#6001.

The same function is vendored in two places here, both with the same gap; both are fixed:

• src/megatron/bridge/models/qwen_vl/modelling_qwen3_vl/utils.py
• src/megatron/bridge/models/exaone/exaone45/modelling_exaone45/utils.py

Measured thresholds for the unclamped path (largest valid length that still crashes):

topology	align_size	crashes at valid length	safe from
TP1/CP2	4	1	2
TP2/CP2	8	1, 2	3
TP2/CP4	16	1–6	8
TP4/CP4	32	1–8	16

Larger TP×CP grids are affected at larger valid lengths, so short prompts plus a wide
parallel grid is the general trigger.

How we hit it

Qwen3.6-35B-A3B RL training on H20-141G, THD (use_remove_padding) + TP2/CP2, verl driving
Megatron through the bridge. Our trigger is verl's multi-trajectory divisibility padding row
— a deliberate 2-token no-op sample — but nothing about the crash is specific to that source.

The runtime module inspection was unambiguous: verl's own copy of the function carries the
clamp, both vendored copies here do not, and the traceback lands in the vendored copy.
Causality was established by single-variable isolation: with a longer padding row the same
config trains fine for 4 steps; reverting only the row length reproduces the crash.

Fix

Clamp the first chunk exactly as the remaining chunk already is, and add the matching
max(..., 0) to remain_len. Plus a regression test.

Test

pre-commit run --files <the three changed files> — all hooks pass (ruff, ruff-format,
trailing whitespace, end-of-file).

New unit test test_preprocess_packed_seqs_row_shorter_than_alignment in
tests/unit_tests/models/qwen_vl/modelling_qwen3_vl/test_utils.py, following the existing
test_preprocess_packed_seqs (same 2-GPU skipif, same _setup_parallel_state helper): a
batch with one full-length row and one single-token row. With tp=1/cp=2 the alignment is 4,
so the single-token row is the smallest case that reaches the out-of-range slice on
cp_rank 1. It raises before this change and passes after.

Behaviour of the chunk assignment, reproduced for both variants at TP=CP=2, both CP ranks:

valid  rank  before          after
2      0     ok              ok
2      1     RuntimeError    ok      <- only behaviour change
8      0/1   ok              ok
38     0/1   ok              ok
128    0/1   ok              ok
14000  0/1   ok              ok

Rows long enough to be unaffected are bit-identical, i.e. this is a pure guard:

valid=128    rank=0/1  torch.equal(before, after) = True
valid=4096   rank=0/1  torch.equal(before, after) = True
valid=14000  rank=0/1  torch.equal(before, after) = True

End to end on the cluster: with the crash avoided, THD + CP=2 completes 4 training steps and
reward magnitude matches an otherwise identical CP=1 run (0.797/0.697/0.473/0.472 vs
0.847/0.415/0.483/0.457 — GRPO rollout is sampled, so only magnitude and trend are
comparable). Peak reserved memory 99.2 GB (CP=2) vs 113.9 GB (CP=1) of 141 GB.

Scope note: clamping prevents the crash; it does not make a very short row semantically
meaningful (some CP ranks end up holding only zeros for it). That matches the behaviour of
the already-clamped remaining chunk, and is harmless for the padding-row case that triggered
it here (no gradient, excluded from metrics).

Not a duplicate

Searched this repo for preprocess_packed_seqs, context parallel packed, and
expanded size (PRs, all states). The closest open PR, #3869 ("Stabilize Qwen3-VL packed
SFT"), does touch modelling_qwen3_vl/utils.py, but its hunks are at lines ~200/230/636 and
do not touch the CP chunk assignment — no overlap and no conflict. #4304 is about adding
packed-sequence support for qwen3-omni. Also filed the same fix for the other vendored copy
of this function in ISEEKYAN/mbridge#157.

AI assistance

This change was prepared with AI assistance (Claude). I reviewed every changed line, ran the
checks above myself, and can defend the change end-to-end.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[yaoyu-33]

/ok to test 00e11d1

[kamran-nvidia]

/ok to test 00e11d1

[gaohongkui]

Thanks for the review and the /ok to test, @yaoyu-33 @kamran-nvidia.

The three red checks are a transient GitHub outage, not this change. All three failing pre-flight jobs died on the same line, while querying PR labels (gh pr view --json labels) — a metadata call unrelated to the diff:

HTTP 503: No server is currently available to service your request. (https://api.github.com/graphql)

Jobs 95455789649 / 95455789674 / 95455791179, all at 17:58:5x UTC. Because pre-flight gates everything downstream, the lint / unit-test / copyright jobs were SKIPPED, which is why Nemo_CICD_Test, Install test summary and Copyright check surface as FAILURE (and codecov/patch is still waiting for a status that never got produced).

Evidence that a re-run is all this needs: the fourth workflow (Build/validate/release, run 32052731736) has already been re-run as attempt 2, and its pre-flight passed on this same SHA.

Could someone re-run these three?

• CICD NeMo — 32052731111
• Installation Test — 32052731378
• Copyright check — 32052731258

I've deliberately not pushed anything or used "Update branch", since a new SHA would invalidate the approval and both /ok to test grants. The branch is only 3 commits behind main, and those commits touch unrelated files (conversation_processing.py, gpu_backend.py, verification cards) — no overlap with this PR, so there's nothing to rebase.