[feat] Upstream Attn-QAT Video Diffusion Code - #1225

Closed

RandNMR73 wants to merge 203 commits into

mainfrom

Commits on Dec 20, 2025

+baseline

jzhang38
authored and
RandNMR73
committed
da3f43c
+ fp4 linear + fp4 attn, all with 16-bit bwd

jzhang38
authored and
RandNMR73
committed
85553c2

Commits on Dec 23, 2025

+ generator sage 3

jzhang38
authored and
RandNMR73
committed
6a58c3a
update

jzhang38
authored and
RandNMR73
committed
9161ef6
update

jzhang38
authored and
RandNMR73
committed
8abfe23
stash

jzhang38
authored and
RandNMR73
committed
a1ab4a7
update

jzhang38
authored and
RandNMR73
committed
2a28c08
update

jzhang38
authored and
RandNMR73
committed
032016f
update

jzhang38
authored and
RandNMR73
committed
c47be5c
update

jzhang38
authored and
RandNMR73
committed
fa91001
save

jzhang38
authored and
RandNMR73
committed
130b466
1005 morning

jzhang38
authored and
RandNMR73
committed
84ed3ce
update

jzhang38
authored and
RandNMR73
committed
18aebfa
add real and fake quant precision tests
RandNMR73
committed
c137c9f
fake quant done

jzhang38
authored and
RandNMR73
committed
3a9e4d3
save

jzhang38
authored and
RandNMR73
committed
b628b5f
checkpoint
RandNMR73
committed
ebdb160
nvfp4 utils in progress
RandNMR73
committed
9b12552
add inference repo
RandNMR73
committed
2fa6fc1
fix DeepGEMM path
RandNMR73
committed
fe9e94e
fix DeepGEMM path
RandNMR73
committed
6a7f08e
qat attn in progress + refactor nvfp4 utils
RandNMR73
committed
b06c038
fix import
RandNMR73
committed
d41912c
checkpoint (qat attn in progress)
RandNMR73
committed
9bcc411
fix masking + causal and non-causal logic in qat attn
RandNMR73
committed
3b70702

Commits on Dec 24, 2025

5090 testing
RandNMR73
committed
7afe836
add SageAttn3 with QAT
RandNMR73
committed
8d68a4f
print sageattn file
RandNMR73
committed
3641487
adjust sage3 block size to 64x64
RandNMR73
committed
460df09
adjus quant kernel block size
RandNMR73
committed
52480b2
revert block size to 128x128
RandNMR73
committed
346bbfa
add delta_s back
RandNMR73
committed
8678798
add sageattn3 inference example
RandNMR73
committed
f6080f7
change block size back to 64x64
RandNMR73
committed
f2eb855
disable delta_s
RandNMR73
committed
d0e4417

Commits on Dec 25, 2025

re-enable delta_s since inference is terrible without it
RandNMR73
committed
56faaa3
bigger batch size + no grad around wrapped_flash_attn
RandNMR73
committed
36cdddf
8 gpus
RandNMR73
committed
efcb3c4
6 gpus
RandNMR73
committed
ca3b398
update training script
RandNMR73
committed
a34a6df
modify scripts + launch qat run
RandNMR73
committed
eb160a2

Commits on Dec 26, 2025

test sage3 inference with smoothing q but no delta s
RandNMR73
committed
6c3d878
set per_block_mean=True and modify api
RandNMR73
committed
388f6f1
disable delta s
RandNMR73
committed
dbd5bd8

Commits on Jan 5, 2026

try sage3 finetuning again
RandNMR73
committed
30eeec0

Commits on Jan 7, 2026

try sage3 finetuning with aligned backward pass
RandNMR73
committed
3adfb07

Commits on Jan 13, 2026

add inference script with custom weights
RandNMR73
committed
d76dfa0
change block size to 128x128
RandNMR73
committed
654cc49
fix qat_attn kwargs
RandNMR73
committed
71ecf4a
5090 example
RandNMR73
committed
8d43721

Commits on Jan 22, 2026

a bunch of changes
RandNMR73
committed
4cd1297
add benchmarking script + script changes + fix distillation pipeline
RandNMR73
committed
e21848a
more scripts
RandNMR73
committed
e332dbb
[feat] Add Matrix-Game 2.0 (#938)

H1yori233
authored and
RandNMR73
committed
bdbf327
[bugfix] Added VSA Padding logic (#944)

loaydatrain
authored and
RandNMR73
committed
2b8c937
[misc] Allow manual override of Pipeline class through override_pipeline_cls_name (#945)

SolitaryThinker
authored and
RandNMR73
committed
6c4cb78
[New Model] Hunyuan1.5 (#943)

JerryZhou54
authored and
RandNMR73
committed
fdca8fd
[docs] small fixes (#947)
RandNMR73
committed
b824f95
[feat] add sliding_tile attention triton kernel and ROCM support (#916)

ZiguanWang
authored and
RandNMR73
committed
2f49e50
[rocm] Add rocm fastvideo docker image (#952)

SolitaryThinker
authored and
RandNMR73
committed
f2cced2
Add LongCat T2V (Base, Distillation and Refinement) Support to FastVideo (#883)

authored and
RandNMR73
committed
9211fa8
feat: consolidate attention kernels into unified fastvideo-kernel package (#946)

authored and
RandNMR73
committed
8f4cd10
[kernel] Reorg and fix fastvideo-kernel (#962)

authored and
RandNMR73
committed
254b49c
[kernel] Release fastvideo-kernel v0.2.1 (#963)

SolitaryThinker
authored and
RandNMR73
committed
042c20f
[docs] refactor attention docs (#964)

SolitaryThinker
authored and
RandNMR73
committed
9ba352f
[kernel] Fix docker release build for kernel (#965)

SolitaryThinker
authored and
RandNMR73
committed
1a637e7
[chore] release v0.1.7 (#955)

SolitaryThinker
authored and
RandNMR73
committed
9eabb5d
[feat] Add new feature extractors for fvd (#954)

ketakitank
authored and
RandNMR73
committed
05214b5
[fix]: fix sliding_tile_attn with sdpa(without flash_attn) (#967)

authored and
RandNMR73
committed
6eedc0a
[fix]: fix fastvideo-kernel Rocm build and Dockerfile for Rocm (#968)

authored and
RandNMR73
committed
bf1892b
[fix]: fix STA trition kernel for AMD RDNA archs (#969)

authored and
RandNMR73
committed
9c3b8ac
[misc] Add util script to create diffuser HF repo from custom component weights (#970)

SolitaryThinker
authored and
RandNMR73
committed
1afbf8c
[kernel] add turbodiffusion kernels (#972)

SolitaryThinker
authored and
RandNMR73
committed
6c18dda
[docs]: fix various broken links across the documentation (#979)

kuafou
authored and
RandNMR73
committed
e24ea17
[feat] Support absmax style quantization for FP8 (#981)

XOR-op
authored and
RandNMR73
committed
b894a82
[New Model] Turbodiffusion (#971)

loaydatrain
authored and
RandNMR73
committed
f74d6e7
[feat] support Matrix-Game 2.0 streaming generation (#957)

H1yori233
authored and
RandNMR73
committed
ac013ac
[feat] Support text encoder weight override and quantization (#983)

XOR-op
authored and
RandNMR73
committed
a6b738f
Layer offloading (#966)

Ohm-Rishabh
authored and
RandNMR73
committed
f72ced7
[docs] Update docs and README (#975)

SolitaryThinker
authored and
RandNMR73
committed
0db7e3c
[ci] increase ssim and lora inference test timeout (#985)

SolitaryThinker
authored and
RandNMR73
committed
5dc8625
[chore] release fastvideo-kernel 0.2.2 (#986)

SolitaryThinker
authored and
RandNMR73
committed
9d1cb80
[chore] update wechat QR code (#988)

SolitaryThinker
authored and
RandNMR73
committed
98452d9
Add LongCat-Video I2V and Video Continuation (Base, Distillation and Refinement) Support to FastVideo (#953)

shaoxiongduan
authored and
RandNMR73
committed
ae03169
[misc] pin fastvideo-kernel in .toml file (#989)

SolitaryThinker
authored and
RandNMR73
committed
f795892
[feat] add Turbodiffusion I2V pipeline (#984)

loaydatrain
authored and
RandNMR73
committed
490ba38
[misc] add pin_cpu_memory false for RTX 4090 (#990)

SolitaryThinker
authored and
RandNMR73
committed
cd5c017
[chore] release 0.1.7 (real) (#980)

SolitaryThinker
authored and
RandNMR73
committed
6b30205
[examples] Added longcat-video python api examples (#994)

shaoxiongduan
authored and
RandNMR73
committed
44d32ad
[docs]: add LoRA extraction utilities documentation (#992)

authored and
RandNMR73
committed
7381e19
[bugfix] Add configs for TurboDiffusion T2V/I2V models (#993)

authored and
RandNMR73
committed
3e4b8b2
dit

SolitaryThinker
authored and
RandNMR73
committed
c6dc7e8
Revert "dit"

SolitaryThinker
authored and
RandNMR73
committed
6b9626c
[ci] temporarily disable turbodiffusion ssim test (#1000)

authored and
RandNMR73
committed
5753d92
[CI] Fixed Turbodiffusion I2V CI (#1002)

loaydatrain
authored and
RandNMR73
committed
989f1d5
[feat!] Disable FSDP inference by default (#1001)

XOR-op
authored and
RandNMR73
committed
d8b9d7c
[misc] [bugfix] unpin 'av' in pyproject (#1009)

SolitaryThinker
authored and
RandNMR73
committed
0d6c9a5
[feat] Introduce Cosmos 2.5 Text2World pipeline (#974)

KyleShao1016
authored and
RandNMR73
committed
ae439b1
[CI] SSIM tests optimization: load all model weights from Modal persistent Volume (#958)

authored and
RandNMR73
committed
8c99407
[CI] Fix OOM issues in ssim tests (#1011)

SolitaryThinker
authored and
RandNMR73
committed
f78843c
[Bug Fix] Add autograd wrapper for block-sparse attention in fastvideo-kernel + fix CMake extension linking (#1015)

alexzms
authored and
RandNMR73
committed
0224ea9
[chore] release fastvideo-kernel 0.2.3 (#1018)

SolitaryThinker
authored and
RandNMR73
committed
5ec0e09
[feat] Hooks API and layerwise offloading for all DiTs (#1006)

XOR-op
authored and
RandNMR73
committed
bee1a5a
[kernel] Fix fastvideo-kernel release workflow (#1019)

SolitaryThinker
authored and
RandNMR73
committed
9ba9a3f
[kernel] [bugfix] [ci] bump v0.2.4. Fix STA output handling, TurboDiffusion CUDA norm dtypes for fastvideo-kernel unit tests. (#1020)

authored and
RandNMR73
committed
e2979d6
Added LTX-2 Distilled T2V Generation (#1016)

authored and
RandNMR73
committed
51bef40
fix: SP for hunyuanvideo 1.5 (#1026)

XOR-op
authored and
RandNMR73
committed
aec9c4d
add inference repo
RandNMR73
committed
5121718
fix DeepGEMM path
RandNMR73
committed
ce95729
checkpoint (qat attn in progress)
RandNMR73
committed
900cc8d
rebase
RandNMR73
committed
a7098ad

Commits on Jan 25, 2026

Commits on Jan 26, 2026

update .gitmodules
RandNMR73
committed
c8d3a05

Commits on Jan 27, 2026

updates
RandNMR73
committed
d127e51
update pyproject.toml
RandNMR73
committed
2e7b3f7
update pyrpoject.toml again
RandNMR73
committed
afd6769
ignore fastvideo_kernel
RandNMR73
committed
1f39e5c
update batch inference script
RandNMR73
committed
e57ffd2
fix sage3 api bug
RandNMR73
committed
dd2096a
ignore fastvideo_kernel
RandNMR73
committed
0c128c8
add sage3 fwd + bf16 bwd script
RandNMR73
committed
72b61ab
update benchmark scripts
RandNMR73
committed
75d6cf1
update benchmarking script attention flop counts
RandNMR73
committed
6f532a1
fix 14B validation videos
RandNMR73
committed
643c8e9
update sage3 to use per_block_mean=True
RandNMR73
committed
33a2d51

Commits on Jan 28, 2026

modified sage3 with cfg=3 inference
RandNMR73
committed
fc10481
update
RandNMR73
committed
3aa5b1f
two level P ablation
RandNMR73
committed
0543dbd
smooth k ablation:
RandNMR73
committed
098ee30
test 2 level quant q, k, v
RandNMR73
committed
58cb2f4
use the product of the two sfs
RandNMR73
committed
3ee1357
remove incorrect two level qkv and revert
RandNMR73
committed
9361c72
sage3 inference 1.3B baseline
RandNMR73
committed
96609c0

Commits on Jan 29, 2026

fp4 1.3B inference
RandNMR73
committed
a866163
add fa2 benchmarking
RandNMR73
committed
af03817
add combined benchmarks
RandNMR73
committed
011b9e7
update benchmark scripts
RandNMR73
committed
c1a6d7f
update default num heads
RandNMR73
committed
68002a0

Commits on Feb 14, 2026

vsa with qat training kernel
RandNMR73
committed
061c205

Commits on Apr 8, 2026

clean
RandNMR73
committed
9c3d7e3
Merge branch 'matthew/clean' into sync-branch
RandNMR73
committed
1cdec40
clean
RandNMR73
committed
ad099ca
fix
RandNMR73
committed
bda89a5
clean
RandNMR73
committed
46cb669
clean
RandNMR73
committed
088cc8d
clean
RandNMR73
committed
e3e075a
clean
RandNMR73
committed
12829a6
clean
RandNMR73
committed
764fa94
clean
RandNMR73
committed
7126879
clean
RandNMR73
committed
6278c9b
clean
RandNMR73
committed
84f615b
clean
RandNMR73
committed
0232183
clean
RandNMR73
committed
026ca65
clean
RandNMR73
committed
12639e1

Commits on Apr 9, 2026

clean
RandNMR73
committed
8af2a5b
clean
RandNMR73
committed
027c324
clean
RandNMR73
committed
d96de5f
clean
RandNMR73
committed
5832ad9
fix
RandNMR73
committed
4ed79b9
fix precommit
RandNMR73
committed
43d277e
fix
RandNMR73
committed
3fd5502
Apply suggestion from @gemini-code-assist[bot]

RandNMR73
and
gemini-code-assist[bot]
authored
8a70dd0
fix
RandNMR73
committed
159376f
fix
RandNMR73
committed
a06fb48
fix
RandNMR73
committed
9a6e327
fix
RandNMR73
committed
bf010e8
fix
RandNMR73
committed
6e7d1a9
fix
RandNMR73
committed
54b2182
fix
RandNMR73
committed
1fa1739
fix
RandNMR73
committed
f313af0
fix
RandNMR73
committed
f1e7b2a
fix
RandNMR73
committed
4636a8a
fix
RandNMR73
committed
2e2899f
fix
RandNMR73
committed
e4a7074
fix
RandNMR73
committed
1d3ff73
fix
RandNMR73
committed
24d3f97
fix
RandNMR73
committed
a6a1efd
fix
RandNMR73
committed
5230aaa
fix
RandNMR73
committed
6e1b285
fix
RandNMR73
committed
6ccc603
fix
RandNMR73
committed
5fab762
fix
RandNMR73
committed
eccf1a9
fix
RandNMR73
committed
10d99ea
fix
RandNMR73
committed
bf39c83
fix
RandNMR73
committed
952b702
fix
RandNMR73
committed
2619107
fix
RandNMR73
committed
b46ce2c
fix
RandNMR73
committed
ee775e8
fix
RandNMR73
committed
cb5eb77
fix
RandNMR73
committed
a3a4e04
fix
RandNMR73
committed
88f9ac1
all tests passing
RandNMR73
committed
16d8874
fix
RandNMR73
committed
15bda19
fix
RandNMR73
committed
b978a92
fix
RandNMR73
committed
ca96349
fix
RandNMR73
committed
aafa257
fix
RandNMR73
committed
12a993e

Commits on Apr 10, 2026

fix
RandNMR73
committed
6b42166
fix
RandNMR73
committed
52e3484
fix
RandNMR73
committed
42a292e

Commits on Apr 11, 2026

Update .gitignore
RandNMR73
authored
567dd87
Remove explicit_package_bases from mypy settings
RandNMR73
authored
3f818d0