Repository navigation
Commit dd7464a
authored
Update lora def (pytorch#18342)
### Summary
Update lora def to use nn.linear instead of torch.nn.functional.linear
1. Remove debug string from FQN. Otherwise we get different quant
indices for lora and non-lora (because lora_a is quantized), after this
change, and the foundation weights are no longer shareable.
2. Having nn.Linear as a submodule allows torchao quant to capture it,
and we don't need custom logic filtering LoraLinear. as lora_a and
lora_b weren't captured before, exclude these to maintain current
behavior. Later we can remove this, optionally.
3. Having nn.Linear as submodule also means we need to remap lora
weights, as weight names have an extra 'linear' in them.
4. Add @Property for weight, bias so that it remains BC and can be
treated as a regular linear module. This is used in
[load_weights_from_attention.py](https://github.com/pytorch/executorch/blob/eb92ceca74d0bace6d01115ae7f860b3baed5c2c/examples/models/llama/static_attention.py#L1154
### Test plan
CI1 parent fd1581d commit dd7464a
3 files changed
Lines changed: 28 additions & 35 deletions
File tree
- backends/xnnpack/operators
- examples/models/llama
- source_transformation
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
625 | 625 | | |
626 | 626 | | |
627 | 627 | | |
628 | | - | |
| 628 | + | |
629 | 629 | | |
630 | 630 | | |
631 | 631 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
29 | | - | |
30 | | - | |
31 | | - | |
32 | | - | |
33 | | - | |
34 | | - | |
35 | | - | |
36 | | - | |
37 | | - | |
38 | 29 | | |
| 30 | + | |
39 | 31 | | |
40 | 32 | | |
41 | 33 | | |
42 | 34 | | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
43 | 52 | | |
44 | | - | |
| 53 | + | |
45 | 54 | | |
46 | 55 | | |
47 | 56 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
144 | 144 | | |
145 | 145 | | |
146 | 146 | | |
147 | | - | |
148 | | - | |
149 | | - | |
150 | | - | |
151 | | - | |
152 | | - | |
153 | | - | |
154 | | - | |
155 | | - | |
156 | | - | |
157 | | - | |
158 | | - | |
159 | | - | |
160 | | - | |
161 | | - | |
162 | | - | |
163 | | - | |
164 | | - | |
165 | | - | |
166 | | - | |
167 | | - | |
168 | | - | |
169 | | - | |
170 | | - | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
171 | 155 | | |
172 | 156 | | |
173 | 157 | | |
| |||
0 commit comments