Skip to content

Commit 02b38d6

Browse files
authored
Vulkan: accept a SymInt delegate arg as a plain Int EValue (pytorch#21848)
### Summary `VulkanBackend::execute` required every SymInt input to arrive as a scalar tensor: ```cpp VK_CHECK_COND( args[i]->isTensor(), "Cannot handle symint arg to graph that is not derived from a " "scalar tensor at the moment."); ``` That holds when the value was lifted from a scalar tensor, but not when the Method's schema declares the argument as `SymInt` — the emitter then passes a plain `Int` EValue and the delegate rejects it. A dynamic-shape export whose sequence length crosses the delegate boundary as a SymInt hits this. This PR accepts both forms on the way in, and adds the mirrored handling on the way out, which was missing entirely: the output loop knew about tensors and TensorRefs but not SymInts, so a graph returning one fell through to `"Could not handle output with type"`. An output SymInt is now written into the caller's slot whether that slot is a scalar tensor (`Int` or `Long`) or an `Int` EValue. Behaviour for the existing scalar-tensor path is unchanged. ### Test plan - Existing `backends/vulkan/test/test_vulkan_delegate.py` suite. - On device (Galaxy S26 Ultra, Adreno): Gemma 4 and LFM2.5-VL exports whose decoder takes a dynamic sequence length pass the SymInt across the delegate boundary as a plain `Int` and generate correctly. Without this change the delegate aborts at the `VK_CHECK_COND` above during the first `execute()`. Originally written by @Nklockiewicz and Mateusz Kopciński as part of Gemma 4 Vulkan enablement; extracted here as a standalone change. cc @SS-JIA @manuelcandales @digantdesai @cbilgin Fixes pytorch#21877
1 parent d62d70f commit 02b38d6

1 file changed

Lines changed: 46 additions & 6 deletions

File tree

‎backends/vulkan/runtime/VulkanBackend.cpp‎

Lines changed: 46 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -764,12 +764,26 @@ class VulkanBackend final : public ::executorch::runtime::BackendInterface {
764764
args[i]->toTensor().numel(),
765765
equivalent_scalar_type(args[i]->toTensor().scalar_type()));
766766
} else if (compute_graph->val_is_symint(iref)) {
767-
VK_CHECK_COND(
768-
args[i]->isTensor(),
769-
"Cannot handle symint arg to graph that is not derived from a "
770-
"scalar tensor at the moment.");
771-
bool was_updated = maybe_update_scalar_tensor(
772-
compute_graph, iref, args[i]->toTensor());
767+
// A SymInt input may arrive either as a scalar tensor (the convention
768+
// when the value was lifted from one) or as a plain Int EValue, which
769+
// is what the emitter produces when the Method's schema declares the
770+
// argument as SymInt. Accept both.
771+
bool was_updated = false;
772+
if (args[i]->isTensor()) {
773+
was_updated = maybe_update_scalar_tensor(
774+
compute_graph, iref, args[i]->toTensor());
775+
} else if (args[i]->isInt()) {
776+
const int32_t cur_val = compute_graph->read_symint(iref);
777+
const int32_t new_val = static_cast<int32_t>(args[i]->toInt());
778+
if (new_val != cur_val) {
779+
compute_graph->set_symint(iref, new_val);
780+
was_updated = true;
781+
}
782+
} else {
783+
VK_THROW(
784+
"Could not handle symint arg: caller's EValue is neither "
785+
"a scalar tensor nor an Int.");
786+
}
773787
// Since symint inputs may impact tensor's sizes, trigger a resize if
774788
// any symbolic integer shapes are updated.
775789
should_propagate_resize = should_propagate_resize || was_updated;
@@ -846,6 +860,32 @@ class VulkanBackend final : public ::executorch::runtime::BackendInterface {
846860
args[o]->toTensor().mutable_data_ptr(),
847861
args[o]->toTensor().numel(),
848862
equivalent_scalar_type(args[o]->toTensor().scalar_type()));
863+
} else if (compute_graph->val_is_symint(oref)) {
864+
// Mirror of the input path: write the graph's current SymInt value
865+
// into the caller's slot, which may be either a scalar tensor or a
866+
// plain Int EValue.
867+
const int32_t out_val = compute_graph->read_symint(oref);
868+
if (args[o]->isTensor()) {
869+
executorch::aten::Tensor& dst = args[o]->toTensor();
870+
const auto dtype = dst.scalar_type();
871+
if (dtype == executorch::aten::ScalarType::Int) {
872+
*dst.mutable_data_ptr<int32_t>() = out_val;
873+
} else if (dtype == executorch::aten::ScalarType::Long) {
874+
*dst.mutable_data_ptr<int64_t>() = static_cast<int64_t>(out_val);
875+
} else {
876+
VK_THROW(
877+
"Could not handle symint output with destination tensor dtype ",
878+
static_cast<int>(dtype));
879+
}
880+
} else if (args[o]->isInt()) {
881+
// Overwrite the EValue's int payload in place. EValue stores its int
882+
// as int64_t, so a SymInt fits without truncation.
883+
*args[o] = EValue(static_cast<int64_t>(out_val));
884+
} else {
885+
VK_THROW(
886+
"Could not handle symint output: caller's EValue is neither "
887+
"a scalar tensor nor an Int.");
888+
}
849889
}
850890
// TensorRef values represent constant tensors which will not have been
851891
// modified by the graph execution. Therefore, if a constant tensor is

0 commit comments

Comments
 (0)