Skip to content

Commit 0a6e214

Browse files
committed
[refactor]refactor attention kernel
1 parent b1faaa9 commit 0a6e214

2 files changed

Lines changed: 311 additions & 326 deletions

File tree

fastvideo/attention/ring/kernels/__init__.py

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,15 @@
3535
pytorch_attn_forward,
3636
)
3737

38+
if HAS_FLASH_ATTN:
39+
from flash_attn import flash_attn_func
40+
41+
if HAS_SAGE_ATTENTION:
42+
import sageattention
43+
44+
if HAS_SPARSE_SAGE_ATTENTION:
45+
from spas_sage_attn.autotune import SparseAttentionMeansim
46+
3847

3948
class AttnType(Enum):
4049
AITER = "aiter"

0 commit comments

Comments
 (0)