Skip to content

Commit e9025a9

Browse files
msluszniakSS-JIA
authored andcommitted
Vulkan: size texture-vs-buffer choice by the bound, not the trace hint
filter_invalid_reprs() decides whether a tensor can live in a texture by comparing required_image_extents(tensor_val.shape) against the device limits. For a symbolic dimension that comparison resolves at the dim's *hint* -- the size of the example the model happened to be traced with -- rather than the maximum its exported range allows. So a model traced with a small example picks a texture that is only valid at that size, and silently produces wrong results once it runs larger. Nothing raises. Repro: the Supertonic TTS text encoder, dynamic T in [8, 512], always executed at T=512 on a Galaxy S26 Ultra (Adreno 840). Only the trace-time example length differs; the edge graphs are identical (552 nodes, same ops, same literal args, same symbolic shapes): traced at T= 64 cosine 0.537966 .pte 18147970 bytes traced at T=128 cosine 0.537966 .pte 18147970 bytes (identical file) traced at T=192 cosine 0.999414 .pte 18149890 bytes traced at T=512 cosine 0.999414 .pte 18149890 bytes (identical file) The emitted binaries cluster exactly on the correctness boundary, which is what identifies this as a lowering decision rather than a bad kernel. With this change the T=64 export emits the 18149890-byte artifact and scores 0.999414. Evaluating the bound instead also covers the unbounded case: a dim with no finite upper bound cannot be shown to fit any texture, so it falls back to buffer storage. Measured on Supertonic (CPU reference reproduced by the XNNPACK delegate at cosine 1.000000): text_encoder 0.406303 -> 0.999414 vector_estimator 0.994432 -> 0.999994 vocoder 0.016757 -> 0.999977
1 parent 6a03b0b commit e9025a9

1 file changed

Lines changed: 37 additions & 4 deletions

File tree

‎backends/vulkan/utils.py‎

Lines changed: 37 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1181,6 +1181,33 @@ def make_tensor_repset(tensor_repr: TensorRepr) -> TensorRepSet:
11811181
raise RuntimeError(f"Unsupported storage type {tensor_repr.storage_type}")
11821182

11831183

1184+
def upper_bound_size(dim: Union[int, torch.SymInt]) -> Optional[int]:
1185+
"""Largest value a (possibly symbolic) tensor dimension can take.
1186+
1187+
Returns None if no finite bound is known.
1188+
1189+
A symbolic dim compares against a limit using its *hint* -- the size of the
1190+
example input the model happened to be traced with -- not the maximum the
1191+
exported range allows. Sizing decisions must use the bound instead, or a
1192+
model traced with a small example will make a choice that is invalid once it
1193+
runs at a larger size.
1194+
"""
1195+
if not isinstance(dim, torch.SymInt):
1196+
return int(dim)
1197+
if not dim.node.expr.free_symbols:
1198+
return int(dim.node.expr)
1199+
shape_env = dim.node.shape_env
1200+
if shape_env is None:
1201+
return None
1202+
try:
1203+
upper = shape_env.bound_sympy(dim.node.expr).upper
1204+
except Exception:
1205+
return None
1206+
if upper is None or not upper.is_finite:
1207+
return None
1208+
return int(upper)
1209+
1210+
11841211
def filter_invalid_reprs(
11851212
tensor_val: FakeTensor,
11861213
tensor_repset: TensorRepSet,
@@ -1198,11 +1225,17 @@ def filter_invalid_reprs(
11981225
can be used to produce a valid image texture for the given tensor (i.e. fits within
11991226
texture limits).
12001227
"""
1228+
# Size the texture by what the dimension CAN be, not by the example the
1229+
# model was traced with. An unbounded dim cannot be shown to fit, so it
1230+
# falls back to buffer storage.
1231+
bounds = [upper_bound_size(d) for d in tensor_val.shape]
12011232
valid_texture_layouts = set()
1202-
for memory_layout in tensor_repset.valid_texture_layouts:
1203-
extents = required_image_extents(tensor_val.shape, memory_layout)
1204-
if extents_are_valid(extents, texture_limits):
1205-
valid_texture_layouts.add(memory_layout)
1233+
if all(b is not None for b in bounds):
1234+
max_shape = torch.Size(bounds)
1235+
for memory_layout in tensor_repset.valid_texture_layouts:
1236+
extents = required_image_extents(max_shape, memory_layout)
1237+
if extents_are_valid(extents, texture_limits):
1238+
valid_texture_layouts.add(memory_layout)
12061239

12071240
# High dimensional tensors require buffer storage
12081241
if len(tensor_val.shape) > 4:

0 commit comments

Comments
 (0)