Skip to content

Commit 392e1f0

Browse files
alexzmsloaydatrainH1yori233SolitaryThinker
committed
[feat] QAD 5090: env-gate attention torch.compile via FASTVIDEO_DISABLE_ATTENTION_COMPILE
DistributedAttention.forward (and the VSA subclass) are hard-decorated with @torch.compiler.disable, which keeps attention out of the surrounding torch.compile graph unconditionally. That blocks the inference compile path even after the FP4 linear and SageAttention3 graph-break fixes land, since the attention forward itself can never be traced. Make the disable conditional on FASTVIDEO_DISABLE_ATTENTION_COMPILE: - unset / "1" / "true" (default): keep torch.compiler.disable — current behavior - "0" / "false" / "no" / "off": drop it so attention can fold into the graph The env var is read at import time (decorators are applied at class definition), which is the right granularity for the multiproc spawn path: each worker re-imports and inherits the parent's env. Co-authored-by: Loay Rashid <42599591+loaydatrain@users.noreply.github.com> Co-authored-by: Kaiqin Kong <k1kong@ucsd.edu> Co-authored-by: William Lin <SolitaryThinker@users.noreply.github.com>
1 parent 0a96eb5 commit 392e1f0

1 file changed

Lines changed: 24 additions & 2 deletions

File tree

fastvideo/attention/layer.py

Lines changed: 24 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# SPDX-License-Identifier: Apache-2.0
22

3+
import os
4+
35
import torch
46
import torch.nn as nn
57

@@ -13,6 +15,26 @@
1315
from fastvideo.layers.rotary_embedding import _apply_rotary_emb
1416

1517

18+
def _attention_compile_disabled() -> bool:
19+
"""Whether to keep attention ``forward`` out of the torch.compile graph.
20+
21+
Defaults to ``True`` (the historical behavior: attention runs eager via
22+
``torch.compiler.disable``). Set ``FASTVIDEO_DISABLE_ATTENTION_COMPILE=0``
23+
to let attention be traced/compiled into the surrounding graph.
24+
"""
25+
val = os.environ.get("FASTVIDEO_DISABLE_ATTENTION_COMPILE")
26+
if val is None:
27+
return True
28+
return val.strip().lower() not in ("0", "false", "no", "off", "")
29+
30+
31+
def _maybe_compiler_disable(fn):
32+
"""Apply ``torch.compiler.disable`` unless disabled via env var."""
33+
if _attention_compile_disabled():
34+
return torch.compiler.disable(fn)
35+
return fn
36+
37+
1638
class DistributedAttention(nn.Module):
1739
"""Distributed attention layer.
1840
"""
@@ -56,7 +78,7 @@ def __init__(self,
5678
self.backend = backend_name_to_enum(attn_backend.get_name())
5779
self.dtype = dtype
5880

59-
@torch.compiler.disable
81+
@_maybe_compiler_disable
6082
def forward(
6183
self,
6284
q: torch.Tensor,
@@ -146,7 +168,7 @@ class DistributedAttention_VSA(DistributedAttention):
146168
"""Distributed attention layer with VSA support.
147169
"""
148170

149-
@torch.compiler.disable
171+
@_maybe_compiler_disable
150172
def forward(
151173
self,
152174
q: torch.Tensor,

0 commit comments

Comments
 (0)