You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 4a06a4f
Browse filesBrowse the repository at this point in the historyBrowse files
[ET-VK][q8ta] Fix addmm arg indexing in QuantizedLinearMatch
Pull Request resolved: pytorch#17567
QuantizedLinearMatch always used args[1] for the weight and args[0] for the
input, which is correct for mm(input, weight) and linear(input, weight, bias?)
but wrong for addmm(bias, input, weight) where the weight is at args[2] and the
input is at args[1].
This was exposed by a torchao change (D69887498) that added Linear+BatchNorm
fusion to prepare_pt2e(). The fusion adds a bias to Linear nodes that previously
had none, causing them to decompose to addmm instead of mm in the edge dialect.
The pattern matcher then read the input's per-tensor dequantize scale (a float
literal) as if it were the weight's per-channel scale (a Node), causing an
assertion failure.
The fix determines the correct arg indices based on whether the anchor node is
addmm. The bias handling at args[0] for addmm was already correct.
Authored-by: Claude
When two quantized linears are chained (e.g. in the SceneX prediction
head), the pattern registry processes them in topological order and
applies replacements immediately. The first linear's replacement calls
`replace_all_uses_with`, which rewrites the dq node's input from the
original quantize op to the new q8ta_linear op. When the second linear
is then matched, `maybe_skip_q_dq_arg_chain` traces back through the
dq node and finds the q8ta_linear op instead of the original quantize
op. The code then extracts scale/zp from args[1]/args[2] of that node,
but q8ta_linear has a different args layout than quantize_per_tensor—
args[1]/args[2] are the first linear's INPUT scale/zp, not its OUTPUT
scale/zp. This causes the second linear to wildly misinterpret its
input values, saturating outputs to -128/127.
The fix reads input scale/zp from the dq node's args instead of the
quantize node's args. The dq node always retains the correct scale/zp
because `replace_all_uses_with` only rewrites its input tensor (args[0]),
not the scale/zp args. This is both simpler and more robust than
special-casing the q8ta_linear args layout.
Pull Request resolved: pytorch#17567
QuantizedLinearMatch always used args[1] for the weight and args[0] for the
input, which is correct for mm(input, weight) and linear(input, weight, bias?)
but wrong for addmm(bias, input, weight) where the weight is at args[2] and the
input is at args[1].
This was exposed by a torchao change (D69887498) that added Linear+BatchNorm
fusion to prepare_pt2e(). The fusion adds a bias to Linear nodes that previously
had none, causing them to decompose to addmm instead of mm in the edge dialect.
The pattern matcher then read the input's per-tensor dequantize scale (a float
literal) as if it were the weight's per-channel scale (a Node), causing an
assertion failure.
The fix determines the correct arg indices based on whether the anchor node is
addmm. The bias handling at args[0] for addmm was already correct.
Authored-by: Claude
ghstack-source-id: 343460523
@exported-using-ghexport
Differential Revision: [D93768640](https://our.internmc.facebook.com/intern/diff/D93768640/)
0 commit comments