Skip to content

Commit 9cd0c12

Browse files
Infer XNNPACK conv dimensionality from the weight rank (pytorch#21588)
### Summary Fixes pytorch#10965. ATen lets `stride`, `padding`, `dilation` and `output_padding` be a single value that is broadcast over every spatial dim, so `len(stride)` is not the convolution's dimensionality. `torch.nn.Conv2d(3, 3, 5, [2])` keeps `stride=(2,)` because `_pair` passes an iterable through unchanged, and that reaches the edge graph as `aten.convolution.default` with a length-1 stride on a rank-4 weight. The XNNPACK backend infers dimensionality from that length in three places, so a 2d conv with a length-1 stride is mistaken for a 1d conv: `Conv1dUnsqueezePass` skips a node unless `len(stride) == 1`, so it picked up the 2d conv, unsqueezed the already-4d weight to 5d and then failed inside the fake tensor convolution rule. That is the traceback in the issue. `ConvolutionConfig.check_constraints` rejects 3d convs with `len(conv_stride) > 2`. A `Conv3d` built with `stride=[2]` passes that guard, gets partitioned, and then hits the same unsqueeze path. The dynamic-quant guard right below it has the same problem in reverse: a genuine 2d conv with a length-1 stride is refused for no reason. `Conv2d.define_node` already normalizes `padding` and `output_padding` from length 1 to length 2, but not `stride` or `dilation`, so `check_or_raise(len(stride) == 2)` fires and `dilation[1]` would be out of range. That is the second failure the issue describes, the one reached by adding `dilation=[2]`. The fix takes the dimensionality from the weight rank, which is unambiguous, and normalizes `stride` and `dilation` in the node visitor the same way the two neighbouring params already are. `_get_act_deps` moves to the same weight-rank derivation, which also stops a length-1 `padding` from making a 2d conv look 1d and silently blocking the `constant_pad_nd` fusion. Review order: `_passes/conv1d_unsqueeze_pass.py`, then `partition/config/gemm_configs.py`, then `operators/op_conv2d.py`, then the tests. ### Test plan Added to `backends/xnnpack/test/ops/test_conv2d.py`: - `test_fp32_conv2d_single_element_spatial_params`, transposed and not, covering `stride=(2,)` with `padding=(1,)` - `test_fp32_conv2d_single_element_dilation`, covering `dilation=(2,)` - `test_fp32_conv3d_single_element_stride_doesnt_partition`, asserting a 3d conv with `stride=(2,)` still produces zero delegates I do not have an ExecuTorch C++ runtime built on this machine, so I could not run `run_method_and_compare_outputs`, and the three tests above were not executed as unit tests. I verified the export and lowering half of them directly against this branch and against `upstream/main` at efd6b55, driving the same modules through `torch.export.export` plus `to_edge_transform_and_lower([XnnpackPartitioner()])` and counting delegate and leftover convolution nodes. On `upstream/main`: ``` conv2d stride=[2] -> Exception: An error occurred when running the 'Conv1dUnsqueezePass' pass conv2d stride=[2] dilation=[2] -> Exception: An error occurred when running the 'Conv1dUnsqueezePass' pass conv3d stride=[2] -> Exception: An error occurred when running the 'Conv1dUnsqueezePass' pass ``` The conv3d row is the partitioner half of the bug. Today that conv is accepted by `ConvolutionConfig` and then dies in the same pass, so the "Only support 1D + 2D Conv" guard is not actually holding. On this branch: ``` conv2d stride=(2,) padding=(1,) transpose=True delegates=1 undelegated_conv=0 conv2d stride=(2,) padding=(1,) transpose=False delegates=1 undelegated_conv=0 conv2d stride=(1,) padding=(1,) dilation=(2,) delegates=1 undelegated_conv=0 conv2d stride=2 (unchanged baseline) delegates=1 undelegated_conv=0 conv1d stride=2 (unchanged baseline) delegates=1 undelegated_conv=0 conv3d stride=(2,) delegates=0 undelegated_conv=1 ``` The conv1d and plain conv2d rows are there to show the unsqueeze path and the ordinary 2d path are untouched. Because `Conv1dUnsqueezePass` now resolves the weight node before deciding, I also ran the same check through `XNNPACKQuantizer` plus `prepare_pt2e` and `convert_pt2e`, to exercise the dequant-unwrap branch. All four still fully delegate on this branch: ``` q conv1d baseline delegates=1 undelegated_conv=0 q conv2d stride=[2] delegates=1 undelegated_conv=0 q conv2d baseline delegates=1 undelegated_conv=0 q conv1d per-channel delegates=1 undelegated_conv=0 ``` And because `_get_act_deps` changed how it decides 1d versus 2d, I re-ran the even-kernel `same`-padding cases that exercise the `constant_pad_nd` fusion. The pad and the conv are still both absorbed into one delegate: ``` qs8 conv2d even-kernel same-padding delegates=1 conv=0 pad=0 qs8 conv1d even-kernel same-padding k=4 delegates=1 conv=0 pad=0 qs8 conv1d same-padding k=4 dilation=2 delegates=1 conv=0 pad=0 ``` This PR was authored with AI assistance using Claude Code. cc @GregoryComer @digantdesai @cbilgin @JakeStevens Signed-off-by: Aditya Singh <adisin650@gmail.com>
1 parent bc5dfe7 commit 9cd0c12

4 files changed

Lines changed: 72 additions & 14 deletions

File tree

‎backends/xnnpack/_passes/conv1d_unsqueeze_pass.py‎

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
from typing import Optional
88

99
import torch
10+
from executorch.backends.transforms import get_shape
1011
from executorch.backends.xnnpack._passes.xnnpack_pass import XNNPACKPass
1112
from executorch.backends.xnnpack.utils.quant_utils import (
1213
is_dequant,
@@ -127,15 +128,17 @@ def call(self, graph_module: torch.fx.GraphModule):
127128
for node in node_list:
128129
if node.op == "call_function":
129130
if node.target.__name__ == "aten.convolution.default":
130-
stride = list(node.args[3])
131-
if len(stride) != 1:
132-
# skip conv if it is not 1d
133-
continue
134-
135131
kernel_node = node.args[1]
136132
if is_dequant(kernel_node) and not is_quant(kernel_node.args[0]):
137133
kernel_node = kernel_node.args[0]
138134

135+
# The weight rank is the only reliable indicator of the conv
136+
# dimensionality. stride, padding and dilation may each be a
137+
# single value that ATen broadcasts over every spatial dim.
138+
if len(get_shape(kernel_node)) != 3:
139+
# skip conv if it is not 1d
140+
continue
141+
139142
if not is_param_node(self.exported_program, kernel_node):
140143
raise AssertionError(
141144
"Expected op for convolution weight node to be a get_attr node or a parameter"

‎backends/xnnpack/operators/op_conv2d.py‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -141,8 +141,14 @@ def define_node(
141141
padding = cast(List[int], node.args[4])
142142
dilation = cast(List[int], node.args[5])
143143
output_padding = cast(List[int], node.args[7])
144+
# ATen broadcasts a single spatial value over every spatial dim, so any of
145+
# these may arrive with length 1 for a 2d convolution.
146+
if len(stride) == 1:
147+
stride = stride + stride
144148
if len(padding) == 1:
145149
padding = padding + padding
150+
if len(dilation) == 1:
151+
dilation = dilation + dilation
146152
if len(output_padding) == 1:
147153
output_padding = output_padding + output_padding
148154

‎backends/xnnpack/partition/config/gemm_configs.py‎

Lines changed: 11 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -361,21 +361,23 @@ def check_constraints(self, node: torch.fx.Node, ep: ExportedProgram) -> bool:
361361
if not super().check_constraints(node, ep):
362362
return False
363363

364-
conv_stride = cast(List[int], node.args[3])
365-
if len(conv_stride) > 2:
364+
kernel_node = get_input_node(node, 1)
365+
kernel_shape = get_shape(kernel_node)
366+
# The weight rank is the only reliable indicator of the conv dimensionality.
367+
# stride, padding and dilation may each be a single value that ATen
368+
# broadcasts over every spatial dim.
369+
conv_dim = len(kernel_shape) - 2
370+
if conv_dim > 2:
366371
why(node, "Only support 1D + 2D Conv")
367372
return False # Only support 1D + 2D Conv
368373

369-
kernel_node = get_input_node(node, 1)
370-
kernel_shape = get_shape(kernel_node)
371374
weight_quant_params = QuantParams.from_weights(kernel_node, ep)
372375
groups = cast(int, node.args[8])
373376
is_transpose = node.args[6]
374377

375378
# XNNPACK does not support dynamic quantization convs that are not 2D or are depthwise
376379
if self._detect_precision(node) == ConfigPrecisionType.DYNAMIC_QUANT and (
377-
len(conv_stride) != 2
378-
or is_depthwise_conv(kernel_shape, groups, is_transpose)
380+
conv_dim != 2 or is_depthwise_conv(kernel_shape, groups, is_transpose)
379381
):
380382
why(
381383
node,
@@ -409,9 +411,9 @@ def _get_act_deps(
409411
and act_input.target == exir_ops.edge.aten.constant_pad_nd.default
410412
and len(act_input.users) == 1
411413
):
412-
conv_padding = cast(List[int], node.args[4])
413-
is_1d = len(conv_padding) == 1
414-
is_2d = len(conv_padding) == 2
414+
conv_dim = len(get_shape(get_input_node(node, 1))) - 2
415+
is_1d = conv_dim == 1
416+
is_2d = conv_dim == 2
415417

416418
pad_value = (
417419
cast(float, act_input.args[2]) if len(act_input.args) > 2 else 0.0

‎backends/xnnpack/test/ops/test_conv2d.py‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -173,6 +173,18 @@ def get_inputs(self):
173173
return (torch.randn(2, 2, 4, 4),)
174174

175175

176+
class Conv3d(torch.nn.Module):
177+
def __init__(self, stride):
178+
super().__init__()
179+
self.conv = torch.nn.Conv3d(2, 2, (2, 2, 2), stride=stride)
180+
181+
def forward(self, x):
182+
return self.conv(x)
183+
184+
def get_inputs(self):
185+
return (torch.randn(1, 2, 4, 4, 4),)
186+
187+
176188
class Conv2dDQSeq(torch.nn.Module):
177189
def __init__(self, transpose=False):
178190
super().__init__()
@@ -290,6 +302,41 @@ def test_fp32_conv2d(self) -> None:
290302
for has_bias in (True, False):
291303
self._test(Conv2d(bias=has_bias, transpose=transpose))
292304

305+
def test_fp32_conv2d_single_element_spatial_params(self) -> None:
306+
# ATen broadcasts a single stride/padding/dilation value over every
307+
# spatial dim, so a 2d conv can carry length-1 spatial params.
308+
for transpose in (True, False):
309+
self._test(
310+
Conv2d(
311+
kernel_size=(3, 3),
312+
stride=(2,),
313+
padding=(1,),
314+
transpose=transpose,
315+
)
316+
)
317+
318+
def test_fp32_conv2d_single_element_dilation(self) -> None:
319+
self._test(
320+
Conv2d(
321+
kernel_size=(3, 3),
322+
stride=(1,),
323+
padding=(1,),
324+
dilation=(2,),
325+
)
326+
)
327+
328+
def test_fp32_conv3d_single_element_stride_doesnt_partition(self) -> None:
329+
# XNNPACK has no conv3d support, and a length-1 stride must not make a 3d
330+
# conv look 1d or 2d to the partitioner.
331+
m = Conv3d(stride=(2,))
332+
(
333+
Tester(m, m.get_inputs())
334+
.export()
335+
.to_edge_transform_and_lower()
336+
.check_count({"torch.ops.higher_order.executorch_call_delegate": 0})
337+
.run_method_and_compare_outputs()
338+
)
339+
293340
def test_fp32_conv2d_permute(self) -> None:
294341
for transpose in (True, False):
295342
for perm_order in list(itertools.permutations([0, 1, 2, 3])):

0 commit comments

Comments
 (0)