Repository navigation
Commit a4e65bf
authored
[PyBindings] Make the torch-free Python bindings safe and complete for CUDA inputs and outputs (#24016)
Follow-up to #23999: makes the Python bindings work well without PyTorch
for programs with CUDA inputs and outputs. Each commit fixes one problem
and adds a test.
## What is wrong today, and the fix
1. A NumPy array passed for a GPU input is read as GPU memory, which can
crash. It is now rejected with a clear message.
2. A run can overwrite a planned output another library still reads.
Each `Method` or `Module` run now waits, before and after, for every
device that holds one of its tensors. Input errors name the input.
3. A complex32 DLPack input is rejected. It is now read.
4. `ExecuTorchResult.__dlpack__` rejects the DLPack 1.0 keywords, so
`np.from_dlpack(result, copy=False)` fails. It now takes `max_version`,
`dl_device` and `copy`, can return versioned capsules, and skips the
device wait for `stream=-1`.
5. A raw DLPack capsule is not accepted as an input. Legacy and
versioned capsules now work, and producers get asked for them.
6. `Method.get_attribute()` returns constants as writable memory, which
may be read-only pages. Constants now come back read-only, DLPack
exports of them are copies, and torch output mode gets a copy.
7. A method from a temporary `Program` with ETDump on uses a freed event
tracer. The tracer now lives as long as any method.
8. A caller cannot ask which device a method expects.
`TensorInfo.device()` and `ExecuTorchResult.device` now return `"cpu"`
or `"cuda:N"`.
9. The installed type stub is never found and has errors. It now ships
fixed, as `_C.pyi` with `py.typed`.
10. A delegate output copied to an unplanned host output
(`alloc_graph_output=False`) crashes. The device copy pass now updates
the graph output, so memory planning places it.
11. `executorch.runtime`'s `Method.execute` now takes `clone_outputs`,
so it can skip the copy.
## Testing
Ran on Linux x86_64 with an H100: CUDA tests in both output modes, CPU
tests with and without torch. Each new test failed on a build without
its fix:
- GPU: planned output wait (Method and Module), wait on a result another
library wrote, DLPack keywords and `stream=-1`, GPU inputs with
unplanned host outputs.
- CPU: host buffer check, error messages, complex32, DLPack keywords,
raw capsules, read-only constants, tracer lifetime (crashed before),
device metadata, device copy planning,
`Method.execute(clone_outputs=False)`.
- The wheel check fails without `_C.pyi` or `py.typed`; mypy resolves
the stub from a built package.
Not tested: the wait after a run for inputs updated in place.1 parent b032012 commit a4e65bf
21 files changed
Lines changed: 1183 additions & 310 deletions
File tree
- .ci/scripts/wheel
- exir
- passes
- tests
- extension/pybindings
- test
- runtime
- executor
- test
- test
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
141 | 141 | | |
142 | 142 | | |
143 | 143 | | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
144 | 154 | | |
145 | 155 | | |
146 | 156 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
78 | 78 | | |
79 | 79 | | |
80 | 80 | | |
| 81 | + | |
81 | 82 | | |
82 | 83 | | |
83 | 84 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
| 18 | + | |
18 | 19 | | |
19 | 20 | | |
20 | 21 | | |
| |||
298 | 299 | | |
299 | 300 | | |
300 | 301 | | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
301 | 310 | | |
302 | 311 | | |
303 | 312 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1340 | 1340 | | |
1341 | 1341 | | |
1342 | 1342 | | |
| 1343 | + | |
| 1344 | + | |
| 1345 | + | |
| 1346 | + | |
| 1347 | + | |
| 1348 | + | |
| 1349 | + | |
| 1350 | + | |
| 1351 | + | |
| 1352 | + | |
| 1353 | + | |
| 1354 | + | |
| 1355 | + | |
| 1356 | + | |
| 1357 | + | |
| 1358 | + | |
| 1359 | + | |
| 1360 | + | |
| 1361 | + | |
| 1362 | + | |
| 1363 | + | |
| 1364 | + | |
| 1365 | + | |
| 1366 | + | |
| 1367 | + | |
| 1368 | + | |
| 1369 | + | |
| 1370 | + | |
| 1371 | + | |
| 1372 | + | |
| 1373 | + | |
| 1374 | + | |
| 1375 | + | |
| 1376 | + | |
| 1377 | + | |
| 1378 | + | |
| 1379 | + | |
| 1380 | + | |
| 1381 | + | |
| 1382 | + | |
| 1383 | + | |
1343 | 1384 | | |
1344 | 1385 | | |
1345 | 1386 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
19 | | - | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
20 | 22 | | |
21 | 23 | | |
22 | 24 | | |
| |||
35 | 37 | | |
36 | 38 | | |
37 | 39 | | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
38 | 43 | | |
39 | 44 | | |
40 | 45 | | |
| |||
45 | 50 | | |
46 | 51 | | |
47 | 52 | | |
48 | | - | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
49 | 60 | | |
50 | 61 | | |
51 | 62 | | |
| |||
57 | 68 | | |
58 | 69 | | |
59 | 70 | | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
60 | 77 | | |
61 | 78 | | |
62 | 79 | | |
| |||
87 | 104 | | |
88 | 105 | | |
89 | 106 | | |
90 | | - | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
91 | 111 | | |
92 | 112 | | |
93 | 113 | | |
| |||
105 | 125 | | |
106 | 126 | | |
107 | 127 | | |
108 | | - | |
109 | | - | |
110 | | - | |
111 | | - | |
112 | | - | |
113 | | - | |
114 | | - | |
115 | | - | |
116 | | - | |
117 | | - | |
118 | | - | |
119 | | - | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
120 | 150 | | |
121 | 151 | | |
122 | 152 | | |
| |||
Whitespace-only changes.
0 commit comments