Repository navigation
Commit 6d0b933
authored
embedding_q4gsw: support is_linear_weight packing (pytorch#20639)
Summary:
The WebGPU `embedding_q4gsw` op failed to load
(`Error::DelegateInvalidCompatibility`, error 48):
`WebGPUGraph::build()` threw `is_linear_weight=true is unsupported` and
`WebGPUBackend.cpp` returned error 48 at `load_forward`.
The `quantized_decomposed.embedding_4bit` and torchao fusions in
`backends/vulkan/patterns/quantized_embedding.py` repack the embedding
weight into the 4-bit linear-layer nibble convention (low nibble = even
dim, high nibble = odd dim) and emit `et_vk.embedding_q4gsw.default(...,
is_linear_weight=true)` — always on OSS main, and for tied
(embedding/LM-head-shared) weights via `_detect_tied_linear_weight`. The
Vulkan runtime supports both packings via its compile-time
`_linear_weight` shader variant; the WebGPU runtime rejected `true`
outright, so any model whose embedding took that path failed to
delegate.
This teaches the WebGPU runtime both packings. The handler now forwards
`is_linear_weight` to the shader through the uniform (the spare `_pad`
field is repurposed; struct size unchanged), and `embedding_q4gsw.wgsl`
selects the nibble with `use_high = is_even != is_linear_weight`. The
`is_linear_weight=false` path is byte-identical to before.
Differential Revision: D1102117461 parent e095c57 commit 6d0b933
3 files changed
Lines changed: 17 additions & 15 deletions
File tree
- backends/webgpu/runtime/ops/embedding_q4gsw
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
33 | | - | |
| 33 | + | |
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
| |||
60 | 60 | | |
61 | 61 | | |
62 | 62 | | |
63 | | - | |
| 63 | + | |
| 64 | + | |
64 | 65 | | |
65 | 66 | | |
66 | 67 | | |
| |||
73 | 74 | | |
74 | 75 | | |
75 | 76 | | |
76 | | - | |
77 | | - | |
78 | | - | |
79 | | - | |
80 | 77 | | |
81 | 78 | | |
82 | 79 | | |
| |||
150 | 147 | | |
151 | 148 | | |
152 | 149 | | |
| 150 | + | |
153 | 151 | | |
154 | 152 | | |
155 | 153 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
14 | | - | |
| 14 | + | |
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
| |||
37 | 37 | | |
38 | 38 | | |
39 | 39 | | |
| 40 | + | |
| 41 | + | |
40 | 42 | | |
41 | | - | |
42 | | - | |
| 43 | + | |
| 44 | + | |
43 | 45 | | |
44 | | - | |
| 46 | + | |
45 | 47 | | |
46 | 48 | | |
47 | 49 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
| 16 | + | |
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| |||
28 | 28 | | |
29 | 29 | | |
30 | 30 | | |
31 | | - | |
| 31 | + | |
32 | 32 | | |
33 | 33 | | |
34 | 34 | | |
| |||
54 | 54 | | |
55 | 55 | | |
56 | 56 | | |
| 57 | + | |
| 58 | + | |
57 | 59 | | |
58 | | - | |
59 | | - | |
| 60 | + | |
| 61 | + | |
60 | 62 | | |
61 | | - | |
| 63 | + | |
62 | 64 | | |
63 | 65 | | |
64 | 66 | | |
| |||
0 commit comments