Repository navigation
Commit 4a61ef4
Cover a CUDA model with weights in the wheel smoke test (pytorch#22089)
### Summary
The CUDA rows of the wheel smoke test export one model through the CUDA
delegate and run it. That model is `x + y`, so it has no weights.
A model with no weights needs nothing from the delegate's external data
file, so it runs whether or not that path works. The CUDA delegate does
not put weights in the `.pte`. It writes them to a separate file named
`aoti_cuda_blob.ptd`, and the caller has to hand that file back at load
time. Every runner script in this repository passes it by name. Nothing
in the smoke test loaded one, so a wheel that could not produce or
consume that file would pass the check and ship.
### What this changes
A second model, a small `nn.Linear`, in the same child process:
- export it through `CudaPartitioner` and write both the `.pte` and the
data file,
- assert the program actually carries a CUDA delegate, so a partitioner
that silently declined the model cannot pass this check on the portable
kernels,
- assert the data file is present under the name the runners expect,
- assert the data file is larger than the one the weightless model above
produces, by at least the size of this model's weight matrix,
- load the program with the data file and compare the result against
eager.
Only the x86_64 rows execute this, the same as the existing check. The
aarch64 rows have no GPU on their validation runner and keep skipping
with a printed reason.
### Why the size check is written as a difference
A data file is written even when the model has no weights, and it is not
small. Measured on sm_80, the weightless model above produces a 256-byte
file: a container around an empty payload. So any fixed lower bound
below 256 is satisfied by a file with nothing in it, and a check like
`size >= weight_bytes` for a small model is true no matter what the
delegate did.
Instead the check writes the weightless program's data file in the same
run, which costs nothing because that program is already compiled, and
subtracts. That keeps working if the container format changes size
later, which a hardcoded 256 would not.
Comparing against the sum over `parameters()` would not work either.
Measured on sm_80 with this model, the delegate wrote the 128-byte
weight into the data file and kept the 16-byte bias out, so that sum is
not a bound on what the file has to hold. The check uses the weight
matrix only.
### Test plan
Ran the new child code on a Linux x86_64 machine with an sm_80 GPU,
against a published nightly CUDA wheel. Both models pass:
```
PASS: a CUDA-delegated model ran on sm_80 and matched eager
PASS: a CUDA-delegated model with weights ran on sm_80 from a 384-byte aoti_cuda_blob.ptd carrying 128 bytes of weights, and matched eager
```
The delegate log on that run names a real content hash for the weights
blob, not the hash of an empty one.
Also confirmed the new check is meaningful in both directions.
Loading the same weighted program without the data file gives:
```
[cuda_backend.cpp:1242] weights_blob '...' not found or update fn is null
[cuda_allocator.cpp:87] cudaMemcpy D2H failed: an illegal memory access was encountered
RuntimeError: Failed to execute method forward, error: 0x1
```
And if the delegate wrote an empty data file for the weighted model, the
measured numbers give 256 minus 256, a difference of 0, against a
128-byte weight matrix, so the assertion fires. The previous version of
this check did not: it compared the raw file size against the parameter
total, and 256 is already larger than 144, so it would have passed on an
empty file.
---------
Co-authored-by: r <r@e>1 parent bd1e461 commit 4a61ef4
1 file changed
Lines changed: 103 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
320 | 320 | | |
321 | 321 | | |
322 | 322 | | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
323 | 326 | | |
324 | 327 | | |
325 | 328 | | |
326 | 329 | | |
327 | 330 | | |
| 331 | + | |
328 | 332 | | |
329 | 333 | | |
330 | 334 | | |
| |||
357 | 361 | | |
358 | 362 | | |
359 | 363 | | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
| 418 | + | |
| 419 | + | |
| 420 | + | |
| 421 | + | |
| 422 | + | |
| 423 | + | |
| 424 | + | |
| 425 | + | |
| 426 | + | |
| 427 | + | |
| 428 | + | |
| 429 | + | |
| 430 | + | |
| 431 | + | |
| 432 | + | |
| 433 | + | |
| 434 | + | |
| 435 | + | |
| 436 | + | |
| 437 | + | |
| 438 | + | |
| 439 | + | |
| 440 | + | |
| 441 | + | |
| 442 | + | |
| 443 | + | |
| 444 | + | |
| 445 | + | |
| 446 | + | |
| 447 | + | |
| 448 | + | |
| 449 | + | |
| 450 | + | |
| 451 | + | |
| 452 | + | |
| 453 | + | |
| 454 | + | |
| 455 | + | |
| 456 | + | |
360 | 457 | | |
361 | 458 | | |
362 | 459 | | |
| |||
396 | 493 | | |
397 | 494 | | |
398 | 495 | | |
399 | | - | |
| 496 | + | |
400 | 497 | | |
401 | 498 | | |
402 | 499 | | |
403 | 500 | | |
404 | 501 | | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
405 | 507 | | |
406 | 508 | | |
407 | 509 | | |
| |||
0 commit comments