Repository navigation
Commit f1adb8b
[ET-VK] Run weight-only int8 linear on the kernel that still exists
FuseQuantizedOpsTransform rewrites a weight-only int8 linear to
aten._weight_int8pack_mm, op_registry lists it as supported, so the
partitioner takes it. At runtime it dispatched to linear_qcs8w_tiled /
linear_qcs8w_coop, and those shaders are not in the repo: pytorch#14041 replaced
them with linear_q8csw_tiled and a different suffix scheme. Every
weight-only int8 model therefore aborts on its first execute with
Could not find ShaderInfo with name
linear_qcs8w_tiled_texture3d_texture3d_texture2d_texture2d_half_o4x3
aten._weight_int8pack_mm carries the same operands as et_vk.linear_q8csw
minus the bias, so register it on that implementation and drop the 8-bit
half of QuantizedLinearQCSNW.cpp, whose shaders are gone. The 4-bit
et_vk.linear_qcs4w path is left untouched; it references missing shaders
too, but the group-wise kernel is not a drop-in replacement for it.
Tested on a Galaxy S26 Ultra (Adreno 840) with all-mpnet-base-v2 and
multi-qa-mpnet-base-dot-v1 lowered through VulkanQuantizer weight-only
int8, at the 382-token bound and resized down to 128. Cosine against the
fp32 eager reference is 0.998280 / 0.995343 (fp16 for comparison:
0.999972). The int8 file is 133.2 MB against 217.9 MB for fp16, and per
execution it runs 59 ms against 79 ms at 382 tokens, 36 vs 49 at 254,
26 vs 29 at 128 (median of 3 rounds, arm order reversed each round,
warm-up discarded).1 parent afc7957 commit f1adb8b
2 files changed
Lines changed: 40 additions & 116 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
944 | 944 | | |
945 | 945 | | |
946 | 946 | | |
| 947 | + | |
| 948 | + | |
| 949 | + | |
| 950 | + | |
| 951 | + | |
| 952 | + | |
| 953 | + | |
| 954 | + | |
| 955 | + | |
| 956 | + | |
| 957 | + | |
| 958 | + | |
| 959 | + | |
| 960 | + | |
| 961 | + | |
| 962 | + | |
| 963 | + | |
| 964 | + | |
| 965 | + | |
| 966 | + | |
| 967 | + | |
| 968 | + | |
| 969 | + | |
| 970 | + | |
| 971 | + | |
| 972 | + | |
| 973 | + | |
| 974 | + | |
| 975 | + | |
| 976 | + | |
| 977 | + | |
| 978 | + | |
| 979 | + | |
| 980 | + | |
947 | 981 | | |
948 | 982 | | |
949 | 983 | | |
| |||
982 | 1016 | | |
983 | 1017 | | |
984 | 1018 | | |
| 1019 | + | |
985 | 1020 | | |
986 | 1021 | | |
987 | 1022 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
19 | | - | |
20 | | - | |
21 | | - | |
22 | | - | |
23 | | - | |
24 | | - | |
25 | | - | |
26 | | - | |
27 | | - | |
28 | | - | |
29 | | - | |
30 | | - | |
31 | | - | |
32 | | - | |
33 | | - | |
34 | | - | |
35 | | - | |
36 | | - | |
37 | | - | |
38 | | - | |
39 | | - | |
40 | | - | |
41 | | - | |
42 | | - | |
43 | | - | |
44 | | - | |
45 | | - | |
46 | | - | |
47 | 19 | | |
48 | 20 | | |
49 | 21 | | |
| |||
178 | 150 | | |
179 | 151 | | |
180 | 152 | | |
181 | | - | |
182 | | - | |
183 | | - | |
184 | | - | |
185 | | - | |
186 | | - | |
187 | | - | |
188 | | - | |
189 | | - | |
190 | | - | |
191 | | - | |
192 | | - | |
193 | | - | |
194 | | - | |
195 | | - | |
196 | | - | |
197 | | - | |
198 | | - | |
199 | | - | |
200 | | - | |
201 | | - | |
202 | | - | |
203 | | - | |
204 | | - | |
205 | | - | |
206 | | - | |
207 | | - | |
208 | | - | |
209 | | - | |
210 | | - | |
211 | | - | |
212 | | - | |
213 | | - | |
214 | | - | |
215 | | - | |
216 | | - | |
217 | | - | |
218 | | - | |
219 | | - | |
220 | | - | |
221 | | - | |
222 | | - | |
223 | | - | |
224 | | - | |
225 | | - | |
226 | | - | |
227 | | - | |
228 | | - | |
229 | | - | |
230 | | - | |
231 | | - | |
232 | | - | |
233 | | - | |
234 | | - | |
235 | | - | |
236 | | - | |
237 | | - | |
238 | | - | |
239 | | - | |
240 | | - | |
241 | | - | |
242 | | - | |
243 | | - | |
244 | | - | |
245 | | - | |
246 | | - | |
247 | | - | |
248 | | - | |
249 | | - | |
250 | | - | |
251 | | - | |
252 | | - | |
253 | | - | |
254 | | - | |
255 | | - | |
256 | 153 | | |
257 | 154 | | |
258 | 155 | | |
| |||
293 | 190 | | |
294 | 191 | | |
295 | 192 | | |
| 193 | + | |
| 194 | + | |
296 | 195 | | |
297 | 196 | | |
298 | 197 | | |
| |||
389 | 288 | | |
390 | 289 | | |
391 | 290 | | |
392 | | - | |
393 | | - | |
394 | | - | |
395 | | - | |
396 | | - | |
397 | | - | |
398 | | - | |
399 | | - | |
400 | | - | |
401 | | - | |
402 | | - | |
403 | | - | |
404 | 291 | | |
405 | 292 | | |
406 | 293 | | |
| |||
411 | 298 | | |
412 | 299 | | |
413 | 300 | | |
414 | | - | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
415 | 304 | | |
416 | 305 | | |
417 | 306 | | |
| |||
0 commit comments