Repository navigation
Commit df3d715
authored
Qualcomm AI Engine Direct - Change transpose output from NCH1 to NC1W before conv (pytorch#21707)
### Summary
This memory layout has better performance on the HTP backend, and lead
to better performance on prefill model.
#### Prefill performance of third partitioned graph:
| Model | w/o | w/ |
|---------------------|:--------:|:--------:|
| llama3.2 3b instruct| 55924 | 35774 |
cc @cbilgin @psiddh1 parent 43f89fb commit df3d715
1 file changed
Lines changed: 13 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
168 | 168 | | |
169 | 169 | | |
170 | 170 | | |
171 | | - | |
172 | | - | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
173 | 178 | | |
174 | | - | |
175 | | - | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
176 | 185 | | |
177 | 186 | | |
178 | 187 | | |
| |||
0 commit comments