Skip to content

Commit e901324

Browse files
msluszniakSS-JIA
authored andcommitted
Vulkan: give Mali its own matmul M-tile selection
kMinOccupancyThreads is tuned for Adreno-class thread counts. On Mali it picks the 1-row tile for typical encoder shapes, which has the least weight reuse of the three tiles, and the 4-row tile loses outright there. Give Mali a 2048-thread bar and skip the 4-row tile. Other devices keep the existing threshold and tile set, so their selection is unchanged.
1 parent 51cdd12 commit e901324

1 file changed

Lines changed: 18 additions & 4 deletions

File tree

‎backends/vulkan/runtime/graph/ops/impl/Matmul.cpp‎

Lines changed: 18 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -27,19 +27,33 @@ namespace vkcompute {
2727
// selected to increase parallelism.
2828
static constexpr uint32_t kMinOccupancyThreads = 4096;
2929

30+
// Mali saturates at a lower thread count than the threshold above was tuned
31+
// for, so the 4-row tile is never worth its register cost there and the 2-row
32+
// tile clears a lower bar. Leaving Mali on the shared threshold picks the
33+
// 1-row tile for typical encoder shapes, which is the least weight reuse of
34+
// the three.
35+
static constexpr uint32_t kMinOccupancyThreadsMali = 2048;
36+
3037
// Returns the M tile size (1, 2, or 4) to use for the matmul shader. The
31-
// largest tile that produces at least kMinOccupancyThreads thread groups is
32-
// chosen; if even tile_m=1 doesn't meet the threshold, tile_m=1 is used.
38+
// largest tile that produces at least the device's occupancy threshold is
39+
// chosen; if even tile_m=1 doesn't meet it, tile_m=1 is used.
3340
uint32_t pick_matmul_tile_m(ComputeGraph* graph, const ValueRef out) {
3441
uint32_t N = graph->size_at<uint32_t>(-1, out);
3542
uint32_t M = graph->size_at<uint32_t>(-2, out);
3643
uint32_t B = graph->dim_of(out) >= 3 ? graph->size_at<uint32_t>(-3, out) : 1;
3744
uint32_t n_groups = utils::div_up_4(N);
45+
const bool is_mali =
46+
graph->context()->adapter_ptr()->device_type() == vkapi::DeviceType::MALI;
47+
const uint32_t min_threads =
48+
is_mali ? kMinOccupancyThreadsMali : kMinOccupancyThreads;
3849
// Try tile_m = 4, 2, 1 in descending order; pick the first that gives
39-
// enough threads.
50+
// enough threads. Mali skips the 4-row tile entirely.
4051
for (uint32_t tile_m : {4u, 2u, 1u}) {
52+
if (is_mali && tile_m == 4u) {
53+
continue;
54+
}
4155
uint32_t total = n_groups * utils::div_up(M, tile_m) * B;
42-
if (total >= kMinOccupancyThreads) {
56+
if (total >= min_threads) {
4357
return tile_m;
4458
}
4559
}

0 commit comments

Comments
 (0)