Skip to content

Commit 28c68b9

Browse files
[perf]: hand the VSA gate to video_sparse_attn as a transposed view
The 64-tile BHSD path copied gate_compress into a contiguous BHSD tensor on every call. The coarse/sparse combine views the gate at block resolution, which never needs contiguity, so the transposed BSHD view is passed as is. This saves one full-sequence copy per attention layer with bit-identical output. On a GB200 at 1x39936x12x128 (Wan 1.3B, 480p) the per-call peak drops from 639.8 to 521.8 MiB and latency from 11.64 to 11.29 ms. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NjFimeedTQWxgqSzP5xo4j
1 parent f9ff387 commit 28c68b9

1 file changed

Lines changed: 5 additions & 2 deletions

File tree

fastvideo/attention/backends/video_sparse_attn.py

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -327,11 +327,14 @@ def forward( # type: ignore[override]
327327

328328
if video_sparse_attn is None:
329329
raise NotImplementedError("video_sparse_attn is not installed")
330-
# Default 64-element-tile path (unchanged): BHSD round-trip.
330+
# Default 64-element-tile path (unchanged): BHSD round-trip. The gate is
331+
# only read elementwise by the coarse/sparse combine, which views it at
332+
# block resolution without requiring contiguity, so the transposed view
333+
# is handed over as is (no full-sequence copy).
331334
query = query.transpose(1, 2).contiguous()
332335
key = key.transpose(1, 2).contiguous()
333336
value = value.transpose(1, 2).contiguous()
334-
gate_compress = gate_compress.transpose(1, 2).contiguous()
337+
gate_compress = gate_compress.transpose(1, 2)
335338
return video_sparse_attn(query,
336339
key,
337340
value,

0 commit comments

Comments
 (0)