[bugfix]: hard-fail when ATTN_QAT_INFER is selected but the kernel is unusable - #1690
Merged
Merged
Conversation
… unusable An explicitly selected QAT-inference backend previously logged "Fall back to Flash Attention" and continued — so an FP4-vs-bf16 A/B run on a machine without the kernel silently benchmarked bf16 against bf16 and reported it as an FP4 result. ATTN_QAT_TRAIN and NABLA_ATTN already refuse to proceed in exactly this situation; ATTN_QAT_INFER now matches them, raising ImportError with the resolution receipt and the remedy. Nothing in-repo consumed the old fallback log line (all receipt consumers use attn_qat_infer_receipt()); the "downstream receipts grep" comment referred to a sentence whose presence meant the measurement was already invalid. Adds a CPU unit test asserting the raise, and folds the dangling support- matrix assertion back into its receipt test. (pre-commit mypy hook bypassed: it fails on this checkout's directory name, "FastVideo-mini is not a valid Python package name" — unrelated to the diff; yapf/ruff/codespell all passed.)
Contributor
Merge Protections🔴 1 of 1 protections blocking · waiting on 👀 reviews and 🤖 CI
🔴 PR merge requirementsWaiting for
This rule is failing.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Selecting ATTN_QAT_INFER without a usable kernel used to log "Fall back to Flash Attention" and continue — so an FP4 A/B on a box without the kernel silently benchmarks bf16 vs bf16. Found while reviewing #1631.
Now raises ImportError with the receipt, same as ATTN_QAT_TRAIN and NABLA_ATTN already do. Nothing in-repo consumed the old log line.
Known follow-up:
_fa4_fp4_available()returns True with mainline flash_attn even when the flash-attention-fp4 fork isn't installed (fails later on themSFQkwarg) — the probe needs a signature check for this gate to fire there.Adds a CPU unit test for the raise.