[misc] Clean up QAD 5090 example inference scripts

[kevin314]

Purpose

Cleans up and homogenizes the example inference scripts involved in the 5090 QAD release

Changes

• Adds test script for NVFP4 Linear + SA2
• Cleans up TAEHV usage
• Updates model defaults to FastWan-QAD checkpoints
• Use shared naming pattern

Test Plan

Tested on RTX 5090, torch=2.12, CUDA 13.0

#NVFP4 Linear + ATTN_QAT_INFER (Modified sageattn3 backend)
python nvfp4_qat_wan2_1_1_3b.py --taehv-checkpoint /path/to/taehv/taew2_1.pth

#NVFP4 Linear + SAGE_ATTN (sageattn2++)
python nvfp4_sa2_wan2_1_1_3b.py --taehv-checkpoint /path/to/taehv/taew2_1.pth

#FP8 Linear + SAGEATTN
python fp8_wan2_1_1_3b.py --taehv-checkpoint /path/to/taehv/taew2_1.pth  

Test Results

Verified visual quality + reported speeds

Checklist

• I ran pre-commit run --all-files and fixed all issues
• I added or updated tests for my changes
• I updated documentation if needed
• I considered GPU memory impact of my changes

For model/pipeline changes, also check:

• I verified SSIM regression tests pass
• I updated the support matrix if adding a new model

[mergify]

Merge Protections

🔴 1 of 1 protections blocking · waiting on 👀 reviews and 🤖 CI

	Protection	Waiting on
🔴	PR merge requirements	👀 reviews and 🤖 CI

🔴 PR merge requirements

Waiting for

• #approved-reviews-by>=1
• check-success=full-suite-passed

This rule is failing.

• #approved-reviews-by>=1
• check-success=full-suite-passed
• check-success=fastcheck-passed
• check-success~=pre-commit
• title~=(?i)^\[(feat|feature|bugfix|fix|refactor|perf|ci|doc|docs|misc|chore|kernel|new.?model|skill|skills|infra)\]

[mergify]

This PR has merge conflicts with the base branch. Please rebase:

git fetch origin main
git rebase origin/main
# Resolve any conflicts, then:
git push --force-with-lease

[gemini-code-assist]

Code Review

This pull request refactors and updates several inference optimization scripts (FP8, NVFP4 QAT, and a new SageAttention 2 script) to use a unified TaehvDecoder class and the updated PipelineConfig system, while also updating project dependencies in pyproject.toml and adjusting a CUDA kernel. The review feedback highlights critical issues that need to be addressed: a typo in the PyTorch dependency index name in pyproject.toml, missing return_frames parameters in generator requests across multiple scripts when TAEHV is enabled (which will cause runtime crashes), and an incorrect CUDA architecture format (12.0a instead of 120a) in the environment configuration for Blackwell GPUs.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[kevin314]

return_frames defaults to True in OutputConfig

• fastvideo/api/schema.py:169
• fastvideo/api/sampling_param.py:172

[SolitaryThinker]

Maintainer review + approved fix wave pushed to this branch.

Review findings (5 majors, condensed):

1. nvfp4_sa2_wan2_1_1_3b.py docstring claimed sm89+ (H100/L40S/4090), but its default path is flashinfer's cutlass FP4 gemm with no fallback — Blackwell sm_120a only; sm89+ is only true for --bf16.
2. nvfp4_qat_wan2_1_1_3b.py docstring claimed "Blackwell (B200/B300, sm100a+)" — the sm_120a cubins do not run on sm100; the correct requirement is RTX 5090-class sm_120a.
3. Both nvfp4 scripts hard-assigned FLASHINFER_CUDA_ARCH_LIST="12.0a", clobbering any user setting and baking in one arch.
4. The TaehvDecoder ported from FastWan_QAD_TAEHV.py flipped decode_video(parallel=True) to False — slower decode than the configuration that produced the published numbers.
5. Deleting FastWan_QAD_TAEHV.py dropped the 5-warmup / 20-run min/max/mean/std benchmark machinery that backed our numbers; plus the fp8 script defaults to SAGE_ATTN without documenting the sageattention dependency, and the nvfp4 scripts died in opaque flashinfer ValueErrors on non-Blackwell GPUs.

Owner directive: keep this cleanup exactly as shaped — the deletion stands and the examples stay lean single-run examples — but separate the benchmark machinery out as its own standalone script rather than embedding multi-run stats into the examples.

Pushed commits:

• f77d1e2 [bugfix]: correct hardware claims and arch handling in QAD 5090 example scripts — fixes both nvfp4 docstrings (default mode = sm_120a / RTX 5090-class; --bf16 is the portable path); switches FLASHINFER_CUDA_ARCH_LIST to setdefault with the value derived from the local GPU (fallback 12.0a); adds a cheap preflight capability gate to both nvfp4 default paths that exits with a one-line error naming the sm_120a requirement and the --bf16 escape hatch (shared helpers in a minimal _qad_common.py); restores parallel=True in all TAEHV decode calls; adds the pip install sageattention note to the fp8 docstring. docs/inference/optimizations.md was checked — it references only current script names and already says sm_120 for the QAT path, so it was left untouched.
• aa4b6b0 [feat]: standalone QAD 5090 benchmark harness (multi-run stats) — new examples/inference/optimizations/benchmark_qad_wan2_1_1_3b.py with --mode {fp8,nvfp4_qat,nvfp4_sa2}, --warmups (default 5), --runs (default 20). Mirrors the corresponding example's pipeline setup per mode and ports the deleted FastWan_QAD_TAEHV.py machinery: request-log silencing, warmup loop, timed runs (denoise + TAEHV decode split), no disk I/O inside the timing loop, and the min/max/mean/std stats table.

Validation: python3 -m py_compile passes on all five touched files. pre-commit run --files ... reports "(no files to check)" — examples/ is excluded by .pre-commit-config.yaml, which is expected. No GPU on the fix machine, so generation was not run; the harness is a faithful port of the previously-working script's loop around the examples' known-working setup.

[mergify]

Pre-commit checks failed

Hi @kevin314, the pre-commit checks have failed. To fix them locally:

# Install pre-commit if you haven't already
uv pip install pre-commit
pre-commit install

# Run all checks and auto-fix what's possible
pre-commit run --all-files

Common fixes:

• yapf: yapf -i <file> (formatting)
• ruff: ruff check --fix <file> (linting)
• codespell: codespell --write-changes <file> (spelling)

After fixing, commit and push the changes. The checks will re-run automatically.

For future commits, pre-commit will run automatically on changed files before each commit.

[SolitaryThinker]

CI note on the current reds — neither is caused by this PR's content:

• pre-commit (exit 127): this branch was cut before [ci]: gate the full-suite trigger on pre-commit and docs build #1572 added .github/scripts/test_gate_full_suite.sh; the workflow (pull_request_target) runs main's workflow file against the PR head, which lacks the script. Known signature; fixed by a rebase.
• Rebasing right now would trade that for two known microscope-unit-tests failures (main's [perf]: register a real backward for FA2 default + masked/varlen custom ops (training-under-compile) #1388/[attn] Make FA4 explicit opt-in via FASTVIDEO_FA4 and delete the runtime fallback machinery #1540 resolver-test conflict, fix staged on maint/fix-resolver-test-post-1388).

Plan: one rebase once the resolver-test fix lands on main, which clears both in a single pass. Holding until then.

[SolitaryThinker]

Rebased onto current main to pick up #1572's full-suite gate script and #1597's resolver-test fix; no content changes (range-diff clean).

[SolitaryThinker]

/test full

[mergify]

This PR has merge conflicts with the base branch. Please rebase:

git fetch origin main
git rebase origin/main
# Resolve any conflicts, then:
git push --force-with-lease

[mergify]

This PR has merge conflicts with the base branch. Please rebase:

git fetch origin main
git rebase origin/main
# Resolve any conflicts, then:
git push --force-with-lease