Skip to content

Commit 6a03b0b

Browse files
msluszniakSS-JIA
andauthored
[ET-VK] Classify the conv2d method by weight shape in conv2d_local_wg_size (pytorch#22051)
### Summary Fixes pytorch#21942. `conv2d_local_wg_size()` picked the convolution method from the shader name alone, and the condition it used matched every conv2d shader: ```cpp if (kernel_name.find("conv2d_pw") != npos || (kernel_name.find("conv2d") != npos && kernel_name.find("conv_transpose2d") == npos)) { method = Conv2dMethod::Pointwise; } else { method = Conv2dMethod::SlidingWindow; } ``` The sliding window shader is itself named `conv2d`, so it matched and was labelled `Pointwise`, making the `SlidingWindow` branch unreachable for every conv2d variant. Only `conv_transpose2d` reached the `else`. The sibling `conv2d_global_wg_size()` directly above uses the identical outer name test but then disambiguates by inspecting the weight's spatial extent, so the two could disagree about the same dispatch: the global size computed as sliding window while the local size was computed as pointwise. This factors that classification into one function used by both, so they cannot drift apart again. ### Why this is a restoration, not a new heuristic The weight-shape check arrived with pytorch#13173, which introduced the tuned `{64 / y, y, 1}` local size for pointwise convolutions. Before that commit the dispatch used `create_local_wg_size(global_size)` for every method: ``` - const utils::uvec3 local_size = graph.create_local_wg_size(global_size); ``` The name test swept sliding window convolutions into the new pointwise size along with the pointwise ones, so this restores the local size they had before pytorch#13173. ### Scope of the behavior change Only sliding window conv2d changes. - **Pointwise** was classified correctly before and after, so it keeps the tuned size. - **Depthwise** is routed to `conv2d_dw_impl()` before this dispatch and never reaches either function. - **Transposed** resolves to the same non-pointwise branch as before (`conv_transpose2d` does not contain the substring `conv2d`, so it already fell through to `create_local_wg_size`). `create_local_wg_size()` is pure arithmetic on the global workgroup size, so the exact before/after can be computed on the host. Across 7452 realistic non-pointwise conv2d output shapes (spatial extents 1 to 512, 3 to 2048 output channels): - every case is 64 threads per group, before and after, so occupancy per dispatch is unchanged and nothing exceeds 64 - only the group shape differs, most commonly `{8, 8, 1}` to one of `{8, 4, 2}`, `{4, 8, 2}`, `{2, 8, 4}`, `{4, 2, 8}` For example, a 128x128x64 output has global size `{128, 128, 16}` and goes from `{8, 8, 1}` to `{8, 4, 2}`. Since sliding window convolutions were last benchmarked under `create_local_wg_size` before pytorch#13173, and the pointwise tuning in that commit was not aimed at them, I would expect this to be neutral to positive. I do not have perf numbers across GPUs, so if you would like this validated on a specific device before landing, say which and I will run it. ### Test plan `vulkan_backend` builds clean. The change is a refactor of method classification plus the restored branch; existing conv2d correctness coverage applies unchanged, since the local workgroup size affects scheduling and not results. cc @SS-JIA @manuelcandales @digantdesai @cbilgin Co-authored-by: Sicheng Stephen Jia <ssjia@meta.com>
1 parent 43afe51 commit 6a03b0b

1 file changed

Lines changed: 37 additions & 27 deletions

File tree

‎backends/vulkan/runtime/graph/ops/impl/Convolution.cpp‎

Lines changed: 37 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -350,6 +350,38 @@ GlobalWorkGrid create_conv2d_gwg(
350350
}
351351
}
352352

353+
// Determines which convolution method a dispatch uses.
354+
//
355+
// Depthwise and transposed convolutions have shader names of their own, but
356+
// the name alone cannot separate pointwise from sliding window: the sliding
357+
// window shader is itself named "conv2d", and a pointwise convolution also
358+
// takes that name when its weights are prepacked. Those two are therefore
359+
// separated by the weight's spatial extent. Shared by the global and local
360+
// workgroup size functions below so that the two cannot disagree about the
361+
// same dispatch.
362+
Conv2dMethod infer_conv2d_method_from_shader(
363+
ComputeGraph* graph,
364+
const vkapi::ShaderInfo& shader,
365+
const ValueRef weight_data) {
366+
const std::string& kernel_name = shader.kernel_name;
367+
// Checked before the plain "conv2d" test below, which "conv2d_dw" and
368+
// "conv2d_pw" would otherwise match too.
369+
if (kernel_name.find("conv2d_dw") != std::string::npos) {
370+
return Conv2dMethod::Depthwise;
371+
}
372+
if (kernel_name.find("conv2d_pw") != std::string::npos) {
373+
return Conv2dMethod::Pointwise;
374+
}
375+
if (kernel_name.find("conv_transpose2d") != std::string::npos) {
376+
return Conv2dMethod::Transposed;
377+
}
378+
const auto& weight_sizes = graph->get_tref(weight_data)->sizes;
379+
if (weight_sizes.at(2) == 1 && weight_sizes.at(3) == 1) {
380+
return Conv2dMethod::Pointwise;
381+
}
382+
return Conv2dMethod::SlidingWindow;
383+
}
384+
353385
// Custom global workgroup size function for conv2d
354386
GlobalWorkGrid conv2d_gwg(
355387
ComputeGraph* graph,
@@ -359,23 +391,8 @@ GlobalWorkGrid conv2d_gwg(
359391
const ValueRef out = args.at(0).refs.at(0);
360392
const ValueRef weight_data = resize_args.at(0);
361393

362-
// Determine method from shader name
363-
Conv2dMethod method;
364-
if (shader.kernel_name.find("conv2d_pw") != std::string::npos ||
365-
(shader.kernel_name.find("conv2d") != std::string::npos &&
366-
shader.kernel_name.find("conv_transpose2d") == std::string::npos)) {
367-
// Check if it's pointwise by examining weight sizes
368-
const auto& weight_sizes = graph->get_tref(weight_data)->sizes;
369-
if (weight_sizes.at(2) == 1 && weight_sizes.at(3) == 1) {
370-
method = Conv2dMethod::Pointwise;
371-
} else {
372-
method = Conv2dMethod::SlidingWindow;
373-
}
374-
} else if (shader.kernel_name.find("conv_transpose2d") != std::string::npos) {
375-
method = Conv2dMethod::Transposed;
376-
} else {
377-
method = Conv2dMethod::SlidingWindow;
378-
}
394+
const Conv2dMethod method =
395+
infer_conv2d_method_from_shader(graph, shader, weight_data);
379396

380397
// Determine stride_equals_dilation from shader name
381398
bool stride_equals_dilation =
@@ -404,17 +421,10 @@ LocalWorkGroup conv2d_lwg(
404421
const std::vector<ArgGroup>& args,
405422
const std::vector<ValueRef>& resize_args) {
406423
(void)args;
407-
(void)resize_args;
408424

409-
// Determine method from shader name
410-
Conv2dMethod method;
411-
if (shader.kernel_name.find("conv2d_pw") != std::string::npos ||
412-
(shader.kernel_name.find("conv2d") != std::string::npos &&
413-
shader.kernel_name.find("conv_transpose2d") == std::string::npos)) {
414-
method = Conv2dMethod::Pointwise;
415-
} else {
416-
method = Conv2dMethod::SlidingWindow;
417-
}
425+
const ValueRef weight_data = resize_args.at(0);
426+
const Conv2dMethod method =
427+
infer_conv2d_method_from_shader(graph, shader, weight_data);
418428

419429
if (method == Conv2dMethod::Pointwise) {
420430
uint32_t lwg_y = 1;

0 commit comments

Comments
 (0)