Repository navigation
Commit 46f9696
authored
Qualcomm AI Engine Direct - Pass migration - part 1 (pytorch#20738)
## Summary
This is **part 1** of a series whose goal is to deprecate
`to_edge_transform_and_lower_to_qnn` and migrate Qualcomm-specific
passes from the export pipeline into `to_edge_transform_and_lower`.
The core change is in `exir/program/_program.py`: after the
`EdgeProgramManager` finishes running `transform_passes`, we now invoke
`lift_constant_tensor_pass` on each method's `ExportedProgram`. Several
QNN passes (`CanonicalizeConv`, `ConvertLinearToConv2d`, etc.) rewrite
constant values and insert new `get_attr` nodes to store them. Without
lifting, these mutated constants are not reflected back into the
`ExportedProgram`; `lift_constant_tensor_pass` promotes the `get_attr`
nodes into placeholders so the modified constants are correctly
propagated.
## Changes
- `exir/program/_program.py`: run `lift_constant_tensor_pass` over every
method after `edge_manager.transform(transform_passes)` completes.
- Move `CanonicalizeConv` and `ConvertLinearToConv2d` out of the export
pipeline (`get_export_passes`) and into the to-edge transform pipeline.
- Add new pass `AnnotateGetAttr` to repopulate `QCOM_QUANT_ATTRS` for
`get_attr` nodes. Passes such as `I64toI32` and `LayoutTransform`
reconstruct the `GraphModule` (`graph_module =
super().call(graph_module).graph_module`), which drops the quant
attributes previously stored on `get_attr` node metadata.
## Test plan
- `CanonicalizeConv`
Test cases:
`TEST_MATRIX = {TestQNNFloatingPointOperator.test_qnn_backend_conv1d
TestQNNFloatingPointOperator.test_qnn_conv1d_batch_norm
TestQNNFloatingPointOperator.test_qnn_backend_conv2d
TestQNNFloatingPointOperator.test_qnn_backend_conv3d_sequential
TestQNNFloatingPointOperator.test_qnn_backend_conv_transpose1d
TestQNNFloatingPointOperator.test_qnn_backend_conv_transpose2d
TestQNNFloatingPointOperator.test_qnn_backend_conv_transpose3d
TestQNNFloatingPointModel.test_qnn_backend_conv1d_relu_log_softmax
TestQNNQuantizedOperator.test_qnn_backend_conv1d
TestQNNQuantizedOperator.test_qnn_conv1d_batch_norm
TestQNNQuantizedOperator.test_qnn_backend_conv2d
TestQNNQuantizedOperator.test_qnn_backend_conv3d_sequential
TestQNNQuantizedOperator.test_qnn_backend_conv_transpose1d
TestQNNQuantizedOperator.test_qnn_backend_conv_transpose2d
TestQNNQuantizedOperator.test_qnn_backend_conv_transpose3d
TestQNNQuantizedModel.test_qnn_backend_conv1d_relu_log_softmax}`
Command:
`python backends/qualcomm/tests/test_qnn_delegate.py ${TEST_MATRIX}
--build_folder build-android/ --host ${HOST_NAME} --device ${DEVICE_ID}
--soc_model ${SOC_ID} --seed 1126 --backend htp`
`python -m pytest
"backends/qualcomm/tests/rework/passes/test.py::test_canonicalize_conv[htp]"
-v`
`python -m pytest
"backends/qualcomm/tests/rework/passes/test.py::test_canonicalize_conv[lpai]"
-v`
- `ConvertLinearToConv2d`
Test cases:
`TEST_MATRIX = {TestQNNFloatingPointOperator.test_qnn_backend_linear
TestQNNFloatingPointOperator.test_qnn_backend_linear_to_conv2d
TestQNNFloatingPointOperator.test_qnn_backend_linear_shared_weights
TestQNNFloatingPointOperator.test_qnn_backend_linear_to_conv2d_shared_weights
TestQNNQuantizedOperator.test_qnn_backend_linear
TestQNNQuantizedOperator.test_qnn_backend_linear_to_conv2d
TestQNNQuantizedOperator.test_qnn_backend_linear_shared_weights
TestQNNQuantizedOperator.test_qnn_backend_linear_to_conv2d_shared_weights
TestQNNQuantizedOperator.test_qnn_backend_linear_block
TestQNNQuantizedOperator.test_qnn_backend_linear_to_conv2d_block
TestQNNQuantizedOperator.test_qnn_backend_linear_qat}`
Command:
`python backends/qualcomm/tests/test_qnn_delegate.py ${TEST_MATRIX}
--build_folder build-android/ --host ${HOST_NAME} --device ${DEVICE_ID}
--soc_model ${SOC_ID} --seed 1126 --backend htp`
`python -m pytest
"backends/qualcomm/tests/rework/passes/test.py::test_convert_linear_to_conv2d[htp]"
-v`
`python -m pytest
"backends/qualcomm/tests/rework/passes/test.py::test_convert_linear_to_conv2d[lpai]"
-v`
`python -m pytest
"backends/qualcomm/tests/rework/htp/op/v68/test.py::test_linear_shared_weights"
-v`
- `AnnotateGetAttr`
Command:
`python -m pytest
"backends/qualcomm/tests/rework/passes/test.py::test_annotate_get_attr"
-v`1 parent 7454b89 commit 46f9696
21 files changed
Lines changed: 957 additions & 474 deletions
File tree
- backends/qualcomm
- _passes
- backends
- gpu
- htp
- lpai
- builders
- tests
- rework
- htp/op/v68
- passes
- src
- utils
- exir/program
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| 8 | + | |
8 | 9 | | |
9 | 10 | | |
10 | 11 | | |
| |||
73 | 74 | | |
74 | 75 | | |
75 | 76 | | |
| 77 | + | |
76 | 78 | | |
77 | 79 | | |
78 | 80 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
55 | 55 | | |
56 | 56 | | |
57 | 57 | | |
58 | | - | |
59 | | - | |
60 | | - | |
61 | | - | |
62 | | - | |
63 | | - | |
64 | 58 | | |
65 | 59 | | |
66 | 60 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
41 | | - | |
42 | | - | |
43 | | - | |
44 | | - | |
| 41 | + | |
45 | 42 | | |
46 | 43 | | |
47 | 44 | | |
48 | | - | |
| 45 | + | |
49 | 46 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
42 | 42 | | |
43 | 43 | | |
44 | 44 | | |
45 | | - | |
46 | | - | |
47 | | - | |
48 | | - | |
| 45 | + | |
49 | 46 | | |
50 | 47 | | |
51 | 48 | | |
52 | | - | |
| 49 | + | |
53 | 50 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
70 | 70 | | |
71 | 71 | | |
72 | 72 | | |
73 | | - | |
74 | | - | |
75 | | - | |
76 | | - | |
| 73 | + | |
77 | 74 | | |
78 | 75 | | |
79 | 76 | | |
80 | | - | |
| 77 | + | |
81 | 78 | | |
82 | 79 | | |
83 | 80 | | |
| |||
0 commit comments