Skip to content

Commit 4bb7bf0

Browse files
ssjiaSS-JIA
authored andcommitted
[ET-VK][qconv] Pad weight_sums buffer to multiple-of-4 alignment
Pull Request resolved: pytorch#17505 The q8ta convolution shaders read weight_sums via ivec4 loads (4 int32 values at once), requiring the buffer to have at least align_up_4(OC) elements. The weight tensor, weight_scales, and bias are all padded via align_width_and_update_state_dict, but weight_sums was created as a 1D tensor of shape (OC,) without any padding. For OC values that are not a multiple of 4 (e.g. OC=1 in the final pointwise conv of MetaNet GreenScreen), this results in out-of-bounds GPU buffer reads. On host testing with ASAN, this manifests as a heap-buffer-overflow. Fix by padding sum_per_output_channel to align_up_4(OC) before creating the constant placeholder. Also fix the C++ test utility compute_weight_sums() which was incorrectly shrinking a pre-allocated aligned buffer. ghstack-source-id: 342806075 @exported-using-ghexport Differential Revision: [D93511633](https://our.internmc.facebook.com/intern/diff/D93511633/)
1 parent 2e8e5a0 commit 4bb7bf0

3 files changed

Lines changed: 37 additions & 1 deletion

File tree

‎backends/vulkan/patterns/quantized_convolution.py‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -226,6 +226,16 @@ def make_q8ta_conv2d_custom_op(
226226
sum_per_output_channel = (
227227
weight_tensor.sum(dim=1).to(torch.int32).contiguous()
228228
)
229+
# Pad weight sums to align OC to multiple of 4, matching the alignment
230+
# applied to weight, weight_scales, and bias above. Without this, the
231+
# GPU shader would read out-of-bounds when OC is not a multiple of 4.
232+
oc = sum_per_output_channel.shape[0]
233+
if oc % 4 != 0:
234+
num_padding = 4 - (oc % 4)
235+
sum_per_output_channel = torch.nn.functional.pad(
236+
sum_per_output_channel, (0, num_padding)
237+
).contiguous()
238+
229239
sums_name = qweight_tensor_name + "_sums"
230240
# Sanitize the name
231241
sums_name = sums_name.replace(".", "_")

‎backends/vulkan/test/custom_ops/test_q8ta_conv2d_pw.cpp‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -210,6 +210,28 @@ static std::vector<TestCase> generate_quantized_conv2d_pw_test_cases() {
210210
}
211211

212212
std::vector<Conv2dConfig> configs = {
213+
// OC < 4 cases to test edge cases with partial output channel blocks
214+
{OutInChannels(1, 16),
215+
InputSize2D(8, 8),
216+
KernelSize(1, 1),
217+
Stride(1, 1),
218+
Padding(0, 0),
219+
Dilation(1, 1),
220+
1},
221+
{OutInChannels(2, 16),
222+
InputSize2D(8, 8),
223+
KernelSize(1, 1),
224+
Stride(1, 1),
225+
Padding(0, 0),
226+
Dilation(1, 1),
227+
1},
228+
{OutInChannels(3, 16),
229+
InputSize2D(8, 8),
230+
KernelSize(1, 1),
231+
Stride(1, 1),
232+
Padding(0, 0),
233+
Dilation(1, 1),
234+
1},
213235
// Pointwise convolutions: kernel size 1x1
214236
{OutInChannels(32, 3),
215237
InputSize2D(64, 64),

‎backends/vulkan/test/custom_ops/utils.cpp‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2064,7 +2064,11 @@ void compute_weight_sums(
20642064
auto& weight_sums_data = weight_sums.get_int32_data();
20652065
auto& quantized_weight_data = quantized_weight.get_int8_data();
20662066

2067-
weight_sums_data.resize(out_features);
2067+
// Don't resize down - the buffer may be pre-allocated with aligned size.
2068+
// Only resize up if needed.
2069+
if (weight_sums_data.size() < static_cast<size_t>(out_features)) {
2070+
weight_sums_data.resize(out_features);
2071+
}
20682072

20692073
// For each output feature, compute the sum of quantized weights
20702074
for (int64_t out_f = 0; out_f < out_features; ++out_f) {

0 commit comments

Comments
 (0)