Skip to content

Commit 94a7536

Browse files
msluszniakSS-JIA
authored andcommitted
Vulkan: use the 1x1 depthwise output tile for 5x5 kernels on Mali too
The 3x3 depthwise shader already drops to a 1x1 output tile on Mali. The 5x5 one kept the default 4x2 tile, so each invocation held 8 accumulators and gathered a separate 25 tap window for every one of them. That spills, and the cost is easy to see next to a 3x3 in the same model: a 5x5 dispatch covering 6000 work items took 4.6 ms while a 3x3 covering 20000 took 0.27 ms, roughly 57x the time per work item for 2.8x the arithmetic. Same rule as 3x3, so Adreno and every other device are untouched. Measured on a Mali-G76 (Galaxy S10+), fp16, interleaved GPU timestamps over 3 to 4 rounds. Depthwise convolution time, median: | model | before | after | | --- | --- | --- | | ssdlite320_mobilenet_v3_large | 19.05 ms | 12.08 ms | | lraspp_mobilenet_v3_large | 15.04 ms | 11.49 ms | | deeplabv3_mobilenet_v3_large | 14.93 ms | 11.52 ms | Ranges do not overlap on any of the three. The individual 5x5 dispatches go from 4.52-4.68 ms to 1.42-1.46 ms (3.2x) and from 0.87-1.12 ms to 0.34-0.44 ms (2.6x). Output is bit identical before and after. (cherry picked from commit a344e8a)
1 parent 1802bcf commit 94a7536

2 files changed

Lines changed: 13 additions & 4 deletions

File tree

‎backends/vulkan/runtime/graph/ops/glsl/conv2d_dw_output_tile.yaml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,3 +32,12 @@ conv2d_dw_output_tile:
3232
OPERATOR: clamp(X, A, B)
3333
BATCH_SIZE_X: 1
3434
BATCH_SIZE_Y: 1
35+
- NAME: conv2d_dw_output_tile_5x5_b1x1
36+
TILE_SIZE: 5
37+
BATCH_SIZE_X: 1
38+
BATCH_SIZE_Y: 1
39+
- NAME: conv2d_dw_output_tile_5x5_b1x1_clamp
40+
OPERATOR: clamp(X, A, B)
41+
TILE_SIZE: 5
42+
BATCH_SIZE_X: 1
43+
BATCH_SIZE_Y: 1

‎backends/vulkan/runtime/graph/ops/impl/Conv2dDW.cpp‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -81,13 +81,13 @@ std::string pick_conv2d_dw_shader(
8181
kernel_name += "_sned";
8282
}
8383

84-
if (is_3x3) {
85-
kernel_name += "_output_tile_3x3";
84+
if (is_3x3 || is_5x5) {
85+
kernel_name += is_3x3 ? "_output_tile_3x3" : "_output_tile_5x5";
86+
// Mali spills with the default 4x2 output tile: each invocation holds 8
87+
// accumulators and loads an overlapping input window for each of them.
8688
if (stride_equals_dilation && graph.device_is_mali()) {
8789
kernel_name += "_b1x1";
8890
}
89-
} else if (is_5x5) {
90-
kernel_name += "_output_tile_5x5";
9191
}
9292

9393
if (clamp_out) {

0 commit comments

Comments
 (0)