You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit d62d70f
Browse filesBrowse the repository at this point in the historyBrowse files
Vulkan: stop delegating nodes past their own support check (pytorch#21847)
### Summary
Two Vulkan backend fixes found while bringing up LFM2.5-VL (a hybrid
attention + gated short-conv VLM) on Android. Neither is specific to
that model.
**1. Nodes were delegated past their own support check.**
Concretely: a rotary embedding's `freqs_cos[input_pos]` feeds the
`hf_rope`
pattern, so its `aten.index.Tensor` was delegated even though
`check_index_tensor_node` rejects a non-1-D `self`. The gather then read
`self[idx, 0, 0, 0]` against a `(4096, 64)` table, and
`resize_index_tensor_node`
set rank-1 sizes on a rank-2 tensor, aborting in `virtual_resize` with
*"new sizes cannot modify the dimensionality of the tensor"*.
This PR excludes the placeholder bindings from the match.
The texture shader still assumes the 1-D form, so `pick_io_storage_fn`
routes a higher-rank `self` to a contiguous buffer.
**2. `VulkanBackend::execute` underflows on a delegate arg-count
mismatch.**
`output_offset = args.size() - num_outputs` is `size_t`. If the
serialized graph
declares more outputs than the delegate call has arguments, that
subtraction
wraps and every output access reads through a wild pointer — the failure
surfaces as a SIGSEGV inside `execute()` with nothing pointing at the
cause.
The reachable route today is a mutated buffer serialized as a graph
output.
LFM2.5's ten short-conv states declared 11 outputs against a 3-argument
call.
This PR checks the counts up front so the result is a readable error
naming both
numbers rather than a segfault. It does not change the mutation handling
itself.
### Test plan
- New `test_vulkan_backend_index_tensor_higher_rank_self` in
`backends/vulkan/test/test_vulkan_delegate.py`, covering
`table[positions]`
with a 2-D table — the shape a rotary embedding uses. This fails before
the
first change and passes after.
- Existing `backends/vulkan/test/test_vulkan_delegate.py` suite.
- On device (Galaxy S26 Ultra, Adreno, driver 0842.19.8): LFM2.5-VL 450M
and
1.6B keep the rotary embedding inside the delegate, lower to a single
Vulkan
call for the decoder, and generate correct text and image responses.
Before
the first change the export aborted in `virtual_resize`; with it, decode
runs
at ~150 tok/s (450M) and ~72 tok/s (1.6B) versus ~100 and ~54 on
XNNPACK, and
text-only TTFT drops from 326→~185 ms and 565→~304 ms respectively.
cc @SS-JIA@manuelcandales@digantdesai@cbilginFixespytorch#21876
0 commit comments