Repository navigation
Commit 575f863
authored
Qualcomm AI Engine Direct - key manager caches by backend and SoC (pytorch#22933)
## Summary
FCB lowering can create multiple QNN backend managers within one
process, each targeting a different SoC. Previously, the manager and
backend-bundle caches used backend type alone as their key, allowing a
manager or device configuration created for one target SoC to be reused
incorrectly for another.
This PR makes those caches FCB-safe by keying them with `(backend type,
SoC)`. Each target receives its own manager and backend bundle, while
repeated requests for the same target continue to reuse the cached
instance.
It also consolidates QNN API compatibility checks in
`QnnSdkCompatibility.h` and applies the shared helper to HTP cache
handling, IR support, and LPAI configuration.
## Compatibility
`PyQnnManager.Compile()` now returns `bytes` rather than a NumPy `uint8`
array. This is an intentional internal binding API alignment: it avoids
the in-tree NumPy-to-bytes conversion, but downstream callers relying on
NumPy-array behavior must adapt. The binding still copies from the QNN
context buffer; this is not a zero-copy optimization.
## Validation
- `lintrunner --skip MYPY -a` on the changed files
- Incremental `build-x86` QNN build
- `pytest -q backends/qualcomm/tests/test_qnn_manager_device_cache.py
backends/qualcomm/tests/test_import_side_effects.py` (60 passed)
## AI assistance
This PR was authored with OpenAI Codex.1 parent e9e21b5 commit 575f863
18 files changed
Lines changed: 150 additions & 66 deletions
File tree
- backends/qualcomm
- aot/python
- runtime/backends
- htp
- ir
- lpai
- tests
- utils
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
245 | 245 | | |
246 | 246 | | |
247 | 247 | | |
248 | | - | |
| 248 | + | |
249 | 249 | | |
250 | 250 | | |
251 | 251 | | |
| |||
254 | 254 | | |
255 | 255 | | |
256 | 256 | | |
257 | | - | |
| 257 | + | |
258 | 258 | | |
259 | 259 | | |
260 | 260 | | |
261 | 261 | | |
262 | 262 | | |
263 | 263 | | |
264 | 264 | | |
265 | | - | |
| 265 | + | |
266 | 266 | | |
267 | | - | |
268 | | - | |
269 | | - | |
270 | | - | |
271 | | - | |
272 | | - | |
273 | | - | |
| 267 | + | |
| 268 | + | |
274 | 269 | | |
275 | 270 | | |
276 | 271 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
138 | 138 | | |
139 | 139 | | |
140 | 140 | | |
141 | | - | |
| 141 | + | |
142 | 142 | | |
143 | 143 | | |
144 | 144 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
| 18 | + | |
18 | 19 | | |
19 | 20 | | |
20 | 21 | | |
| |||
28 | 29 | | |
29 | 30 | | |
30 | 31 | | |
| 32 | + | |
31 | 33 | | |
32 | 34 | | |
33 | 35 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
| 11 | + | |
11 | 12 | | |
12 | 13 | | |
13 | 14 | | |
| |||
50 | 51 | | |
51 | 52 | | |
52 | 53 | | |
53 | | - | |
| 54 | + | |
54 | 55 | | |
55 | 56 | | |
56 | 57 | | |
| |||
66 | 67 | | |
67 | 68 | | |
68 | 69 | | |
69 | | - | |
| 70 | + | |
70 | 71 | | |
71 | 72 | | |
72 | 73 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
| 15 | + | |
15 | 16 | | |
16 | 17 | | |
17 | 18 | | |
18 | 19 | | |
19 | 20 | | |
20 | | - | |
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
50 | 50 | | |
51 | 51 | | |
52 | 52 | | |
| 53 | + | |
| 54 | + | |
53 | 55 | | |
54 | 56 | | |
55 | 57 | | |
| |||
76 | 78 | | |
77 | 79 | | |
78 | 80 | | |
79 | | - | |
| 81 | + | |
80 | 82 | | |
81 | 83 | | |
82 | 84 | | |
| |||
171 | 173 | | |
172 | 174 | | |
173 | 175 | | |
174 | | - | |
175 | | - | |
| 176 | + | |
176 | 177 | | |
177 | 178 | | |
178 | 179 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| 20 | + | |
20 | 21 | | |
21 | 22 | | |
22 | 23 | | |
| |||
109 | 110 | | |
110 | 111 | | |
111 | 112 | | |
112 | | - | |
113 | | - | |
114 | | - | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
115 | 116 | | |
116 | 117 | | |
117 | 118 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| 10 | + | |
| 11 | + | |
10 | 12 | | |
11 | 13 | | |
12 | 14 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| 10 | + | |
| 11 | + | |
10 | 12 | | |
11 | 13 | | |
12 | 14 | | |
| |||
0 commit comments