Skip to content

Commit 65f1260

Browse files
alexzmsWill Lin
authored andcommitted
[bugfix] ltx2: apply video_position_offset_sec RoPE offset
The DiT forward swallowed video_position_offset_sec via **kwargs, so multi-segment rollouts never advanced the temporal RoPE phase between segments, causing ~1s audio/ video desync at each seam. Add the offset to the temporal position coords (mirrors #1422).
1 parent 1e1ac08 commit 65f1260

1 file changed

Lines changed: 7 additions & 0 deletions

File tree

fastvideo/models/dits/ltx2.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2974,6 +2974,13 @@ def forward(
29742974
positions[:, 0, ...] = positions[:, 0, ...] + float(video_position_offset_sec)
29752975
positions = positions.to(hidden_states.dtype)
29762976

2977+
# PR #1422 / will-nbv: honor video_position_offset_sec — shift the video temporal RoPE
2978+
# positions forward (in seconds) to match the audio continuation pre-roll, else the
2979+
# param is swallowed by **kwargs and multi-segment continuation accumulates A/V desync.
2980+
video_position_offset_sec = kwargs.get("video_position_offset_sec", 0.0)
2981+
if video_position_offset_sec != 0.0:
2982+
positions[:, 0, :, :] = positions[:, 0, :, :] + video_position_offset_sec
2983+
29772984
# Pad positions to match padded sequence length for SP cross-attention
29782985
if sp_world_size > 1 and video_padded_seq_len > video_original_seq_len:
29792986
padding_needed = video_padded_seq_len - video_original_seq_len

0 commit comments

Comments
 (0)