Repository navigation
Commit 3dfa644
Fix out-of-bounds traversal in reduce_util for non-contiguous tensors (pytorch#21517)
### Summary
Fixes pytorch#16429.
`dequantize_per_channel` on a channels-last input writes past the end of
its output. The reported symptom is wrong values, but the underlying
problem is a memory error and it is not in the quantized kernel.
`apply_on_flat_ix_with_dim_mask_and_base` in
`kernels/portable/cpu/util/reduce_util.h` walks the elements that map to
one output index. When the innermost dim is exhausted it rewinds
`curr_index` and carries into the next dim, using:
```cpp
curr_index -= strides[curr_dim - 1];
```
The comment above that line states the assumption: it treats
`strides[curr_dim - 1]` as equal to `in.size(curr_dim) *
strides[curr_dim]`. That identity only holds for a contiguous tensor.
For a `(2, 2, 3, 3)` channels-last tensor, strides are `[18, 1, 6, 2]`,
and the two disagree at `curr_dim = 1` (2 vs 18) and `curr_dim = 2` (18
vs 1). The rewind is then wrong and the walk leaves the buffer.
For `dequantize_per_channel` on that tensor the walk covers indices 0 to
17, skips 18 to 34 entirely, and writes 17 times past the end of a 36
element output. Running it under the existing kernel tests produces
wrong values followed by `double free or corruption (!prev)`.
This changes the line to subtract the value the comment describes. For a
contiguous tensor the two expressions evaluate to the same number, so
that path is unaffected, and it is in the carry-over rather than the
inner loop, so it is not on the per-element path.
Most reduction ops do not reach this. `op_amax`, `op_amin`, `op_mean`,
`op_sum` and `op_var` all guard with `tensor_is_default_dim_order(in)`
and reject non-contiguous input before the traversal runs. The ops that
accept it are `op_any`, `op_var_mean` and `dequantize_per_channel`, and
I have confirmed the fault in two of them: `dequantize_per_channel`
writes out of bounds, and `any.dims_out` reads out of bounds and returns
a wrong answer.
I took this over adding a `tensor_is_default_dim_order` guard to
`dequantize_per_channel` to match the other five ops. A guard would stop
the corruption but leaves the traversal wrong for `op_any` and
`op_var_mean`, and pytorch#16429 asks for non-contiguous dim order to be
supported in portable ops rather than rejected. Happy to switch if you
would rather have the guard.
Relaxing the five existing guards is not part of this change.
### Test plan
`OpDequantizeOutTest.DequantizePerChannelChannelsLast` dequantizes a
`(2, 2, 3, 3)` channels-last int8 tensor per channel on `axis=1`. Each
expected value is derived from the element's channel alone, so it passes
only if the
kernel honors the tensor's dim order.
`OpAnyOutTest.ChannelsLastMultiDimReduction` reduces a channels-last
tensor over dims `{0, 2, 3}` with `keepdim=true`. The single true value
sits at channels-last physical index 19, inside the range the old
traversal skipped, so before this change channel 1 was reported false
while channel 0 picked up out-of-bounds memory and was reported true.
Both fail before this change and pass after it.
This also adds `op_dequantize_test.cpp` to
`_quantized_kernels_test_sources` in `kernels/test/CMakeLists.txt`. It
is already in `kernels/quantized/test/targets.bzl`, but the CMake target
lists every other op in that directory and omits this one, so those
tests do not run in a CMake build and the new one would not either.
Full runs after the change: `quantized_kernels_test` 73 passed,
`portable_kernels_test` 1624 ran with 1520 passed and 0 failures.
cc @larryliu0820 @manuelcandales @JakeStevens
---------
Co-authored-by: Jacob Szwejbka <jakeszwe@meta.com>1 parent 52bbb30 commit 3dfa644
5 files changed
Lines changed: 119 additions & 4 deletions
File tree
- kernels
- portable/cpu/util
- test
- quantized/test
- test
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
107 | 107 | | |
108 | 108 | | |
109 | 109 | | |
110 | | - | |
111 | | - | |
112 | | - | |
| 110 | + | |
| 111 | + | |
113 | 112 | | |
114 | | - | |
| 113 | + | |
115 | 114 | | |
116 | 115 | | |
117 | 116 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
432 | 432 | | |
433 | 433 | | |
434 | 434 | | |
| 435 | + | |
| 436 | + | |
| 437 | + | |
| 438 | + | |
| 439 | + | |
| 440 | + | |
| 441 | + | |
| 442 | + | |
| 443 | + | |
| 444 | + | |
| 445 | + | |
| 446 | + | |
| 447 | + | |
| 448 | + | |
| 449 | + | |
| 450 | + | |
| 451 | + | |
| 452 | + | |
| 453 | + | |
| 454 | + | |
| 455 | + | |
| 456 | + | |
| 457 | + | |
| 458 | + | |
| 459 | + | |
| 460 | + | |
| 461 | + | |
| 462 | + | |
| 463 | + | |
| 464 | + | |
| 465 | + | |
| 466 | + | |
| 467 | + | |
| 468 | + | |
| 469 | + | |
| 470 | + | |
| 471 | + | |
| 472 | + | |
435 | 473 | | |
436 | 474 | | |
437 | 475 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
317 | 317 | | |
318 | 318 | | |
319 | 319 | | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
411 | 411 | | |
412 | 412 | | |
413 | 413 | | |
| 414 | + | |
414 | 415 | | |
415 | 416 | | |
416 | 417 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
208 | 208 | | |
209 | 209 | | |
210 | 210 | | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
0 commit comments