Repository navigation
Commit 27d124f
authored
Fix exit safety, package linking and header hygiene in the shared CUDA allocator (pytorch#23499)
Freeing CUDA memory during program exit can access allocator state after
it has been destroyed. Wheel consumers can gain an unnecessary
dependency on the development toolkit's CUDA runtime. The allocator's
public header also carries private bookkeeping that callers never use.
This change keeps the allocator, its rounded pointer map and its pool
table alive until the process exits, so a late free or pool trim from
another library's static destructor still works. Construction is
restricted to `instance()`, and a compile time check confirms that the
backend name is the same class. The wheel package exports the toolkit
include directories without linking another CUDA runtime, adding them
once even when `find_package` runs again. A program that calls the CUDA
runtime itself now links `CUDA::cudart`, which the C++ guide states. The
rounded pointer map moves into the source file, and deallocation skips
its lock while no rounded pointer is live. Alignments above 256 bytes
still use one padded `cudaMalloc`, because retrying a misaligned block
would free it first, and `cudaFree` waits for all work queued on the
device.
Compatibility: `CudaAllocator` can no longer be constructed directly;
use `CudaAllocator::instance()`. Code that used `std::atomic`,
`std::mutex` or `std::unordered_map` through `cuda_allocator.h` must
include those headers itself.
Validation on Linux x86_64 with CUDA 12.8:
- The allocator suite and four other CUDA suites pass. Stand-in tests
check that `allocate` makes one `cudaMalloc` call, never calls
`cudaFree` on the padded path, frees a rounded pointer through its
original block, and refuses a block that misses an alignment of 256 or
less.
- The exit test frees a rounded block and trims the pool from an exit
handler. It passes 20 runs in Release and Debug builds. Reverting the
lifetime of the allocator, its pool table or its pointer map each makes
it fail in an optimized build.
- Compute Sanitizer reports no errors or leaks in the allocator suite.
- Wheel consumers build and run with early and late toolkit discovery,
and an empty consumer records no direct `libcudart` dependency.
Windows and ROCm were not run locally; this pull request's CI covers
both.1 parent 4567831 commit 27d124f
10 files changed
Lines changed: 390 additions & 134 deletions
File tree
- .ci/scripts/wheel
- .github/workflows
- backends/cuda/runtime/test
- docs/source
- extension/cuda
- test
- tools/cmake
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1349 | 1349 | | |
1350 | 1350 | | |
1351 | 1351 | | |
1352 | | - | |
1353 | | - | |
| 1352 | + | |
| 1353 | + | |
| 1354 | + | |
| 1355 | + | |
1354 | 1356 | | |
1355 | 1357 | | |
1356 | 1358 | | |
1357 | 1359 | | |
1358 | 1360 | | |
1359 | 1361 | | |
1360 | 1362 | | |
1361 | | - | |
| 1363 | + | |
| 1364 | + | |
1362 | 1365 | | |
1363 | | - | |
| 1366 | + | |
| 1367 | + | |
| 1368 | + | |
1364 | 1369 | | |
1365 | 1370 | | |
1366 | 1371 | | |
1367 | 1372 | | |
1368 | 1373 | | |
1369 | 1374 | | |
1370 | 1375 | | |
1371 | | - | |
1372 | | - | |
| 1376 | + | |
| 1377 | + | |
| 1378 | + | |
| 1379 | + | |
| 1380 | + | |
| 1381 | + | |
| 1382 | + | |
| 1383 | + | |
| 1384 | + | |
| 1385 | + | |
| 1386 | + | |
| 1387 | + | |
| 1388 | + | |
| 1389 | + | |
| 1390 | + | |
| 1391 | + | |
| 1392 | + | |
| 1393 | + | |
| 1394 | + | |
| 1395 | + | |
| 1396 | + | |
| 1397 | + | |
1373 | 1398 | | |
1374 | 1399 | | |
1375 | 1400 | | |
| |||
1386 | 1411 | | |
1387 | 1412 | | |
1388 | 1413 | | |
| 1414 | + | |
1389 | 1415 | | |
1390 | | - | |
| 1416 | + | |
1391 | 1417 | | |
1392 | 1418 | | |
1393 | 1419 | | |
1394 | 1420 | | |
1395 | | - | |
| 1421 | + | |
| 1422 | + | |
| 1423 | + | |
| 1424 | + | |
| 1425 | + | |
| 1426 | + | |
| 1427 | + | |
| 1428 | + | |
| 1429 | + | |
| 1430 | + | |
| 1431 | + | |
| 1432 | + | |
| 1433 | + | |
1396 | 1434 | | |
1397 | 1435 | | |
1398 | 1436 | | |
| |||
2005 | 2043 | | |
2006 | 2044 | | |
2007 | 2045 | | |
| 2046 | + | |
2008 | 2047 | | |
2009 | 2048 | | |
2010 | 2049 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
79 | 79 | | |
80 | 80 | | |
81 | 81 | | |
82 | | - | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
83 | 86 | | |
84 | 87 | | |
85 | 88 | | |
| |||
1236 | 1239 | | |
1237 | 1240 | | |
1238 | 1241 | | |
1239 | | - | |
1240 | | - | |
| 1242 | + | |
| 1243 | + | |
1241 | 1244 | | |
1242 | 1245 | | |
1243 | 1246 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
577 | 577 | | |
578 | 578 | | |
579 | 579 | | |
580 | | - | |
| 580 | + | |
581 | 581 | | |
582 | 582 | | |
583 | 583 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
| 16 | + | |
16 | 17 | | |
17 | 18 | | |
18 | 19 | | |
| |||
31 | 32 | | |
32 | 33 | | |
33 | 34 | | |
34 | | - | |
35 | | - | |
36 | | - | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
37 | 38 | | |
38 | 39 | | |
| 40 | + | |
39 | 41 | | |
40 | 42 | | |
41 | 43 | | |
42 | 44 | | |
43 | 45 | | |
44 | 46 | | |
45 | | - | |
46 | | - | |
47 | | - | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
48 | 50 | | |
49 | 51 | | |
50 | 52 | | |
| |||
58 | 60 | | |
59 | 61 | | |
60 | 62 | | |
61 | | - | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
62 | 66 | | |
| 67 | + | |
63 | 68 | | |
64 | 69 | | |
65 | 70 | | |
| |||
110 | 115 | | |
111 | 116 | | |
112 | 117 | | |
113 | | - | |
114 | | - | |
115 | | - | |
116 | | - | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
117 | 128 | | |
118 | 129 | | |
119 | 130 | | |
| |||
209 | 220 | | |
210 | 221 | | |
211 | 222 | | |
| 223 | + | |
212 | 224 | | |
213 | 225 | | |
214 | 226 | | |
| |||
235 | 247 | | |
236 | 248 | | |
237 | 249 | | |
| 250 | + | |
238 | 251 | | |
239 | 252 | | |
240 | 253 | | |
241 | 254 | | |
242 | 255 | | |
243 | 256 | | |
| 257 | + | |
244 | 258 | | |
245 | 259 | | |
246 | 260 | | |
| |||
250 | 264 | | |
251 | 265 | | |
252 | 266 | | |
| 267 | + | |
253 | 268 | | |
254 | 269 | | |
255 | 270 | | |
| |||
279 | 294 | | |
280 | 295 | | |
281 | 296 | | |
282 | | - | |
283 | | - | |
284 | | - | |
285 | | - | |
286 | | - | |
287 | | - | |
288 | | - | |
289 | | - | |
290 | | - | |
291 | | - | |
292 | | - | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
293 | 320 | | |
294 | | - | |
295 | | - | |
296 | | - | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
297 | 325 | | |
| 326 | + | |
298 | 327 | | |
299 | | - | |
300 | | - | |
301 | | - | |
302 | | - | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
303 | 338 | | |
304 | | - | |
305 | 339 | | |
306 | | - | |
307 | | - | |
308 | | - | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
309 | 345 | | |
310 | | - | |
| 346 | + | |
| 347 | + | |
311 | 348 | | |
312 | | - | |
313 | | - | |
314 | | - | |
315 | | - | |
316 | | - | |
317 | | - | |
318 | | - | |
319 | | - | |
320 | | - | |
321 | | - | |
322 | | - | |
323 | | - | |
324 | | - | |
325 | 349 | | |
326 | 350 | | |
327 | 351 | | |
328 | | - | |
329 | | - | |
330 | | - | |
331 | | - | |
332 | | - | |
333 | | - | |
334 | | - | |
335 | | - | |
336 | | - | |
337 | | - | |
338 | | - | |
339 | | - | |
340 | | - | |
341 | | - | |
342 | | - | |
343 | | - | |
344 | | - | |
345 | | - | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
346 | 360 | | |
347 | | - | |
348 | | - | |
349 | | - | |
350 | | - | |
351 | | - | |
352 | 361 | | |
353 | 362 | | |
354 | 363 | | |
355 | | - | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
356 | 399 | | |
357 | 400 | | |
358 | 401 | | |
| |||
0 commit comments