Skip to content

Commit ac8bb10

Browse files
Mister-Raggsclaude
andcommitted
[misc] allow ATTN_QAT_TRAIN on Wan cross-attention (train/infer parity)
The cross-attn (attn2) LocalAttention allowlist had ATTN_QAT_INFER (added in hao-ai-lab#1598) but not ATTN_QAT_TRAIN, so QAT finetune silently ran cross-attn in Flash while self-attn fake-quantized — a train/inference mismatch vs the FP4 inference path (which quantizes both). Add ATTN_QAT_TRAIN so the finetune fake-quantizes cross-attn too. (self-attn already had it via the DiT config list.) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 113ef13 commit ac8bb10

1 file changed

Lines changed: 2 additions & 1 deletion

File tree

fastvideo/models/dits/wanvideo.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -186,7 +186,8 @@ def __init__(self,
186186
causal=False,
187187
supported_attention_backends=(AttentionBackendEnum.FLASH_ATTN,
188188
AttentionBackendEnum.TORCH_SDPA,
189-
AttentionBackendEnum.ATTN_QAT_INFER))
189+
AttentionBackendEnum.ATTN_QAT_INFER,
190+
AttentionBackendEnum.ATTN_QAT_TRAIN))
190191

191192
def forward(self, x: torch.Tensor, context: torch.Tensor,
192193
context_lens: int):

0 commit comments

Comments
 (0)