Repository navigation
Commit e11953e
authored
Q4k/Q6k improvements (pytorch#20643)
Small quant improvements:
* Parallelizes writes in Q4k/Q6k kernels.
* Adds support for MLX 6-bit kernel1 parent 6f8a889 commit e11953e
4 files changed
Lines changed: 52 additions & 39 deletions
File tree
- backends/mlx
- builder
- custom_kernel_ops/gguf
- q4k
- q6k
- test
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
157 | 157 | | |
158 | 158 | | |
159 | 159 | | |
160 | | - | |
| 160 | + | |
| 161 | + | |
161 | 162 | | |
162 | 163 | | |
163 | 164 | | |
| |||
533 | 534 | | |
534 | 535 | | |
535 | 536 | | |
536 | | - | |
537 | | - | |
538 | | - | |
539 | | - | |
| 537 | + | |
| 538 | + | |
540 | 539 | | |
| 540 | + | |
541 | 541 | | |
542 | 542 | | |
543 | 543 | | |
| |||
554 | 554 | | |
555 | 555 | | |
556 | 556 | | |
557 | | - | |
558 | | - | |
559 | | - | |
560 | | - | |
561 | | - | |
562 | | - | |
563 | | - | |
564 | | - | |
| 557 | + | |
| 558 | + | |
| 559 | + | |
| 560 | + | |
| 561 | + | |
| 562 | + | |
| 563 | + | |
| 564 | + | |
| 565 | + | |
| 566 | + | |
| 567 | + | |
| 568 | + | |
565 | 569 | | |
566 | 570 | | |
567 | 571 | | |
| |||
654 | 658 | | |
655 | 659 | | |
656 | 660 | | |
657 | | - | |
658 | | - | |
| 661 | + | |
| 662 | + | |
| 663 | + | |
659 | 664 | | |
660 | | - | |
| 665 | + | |
661 | 666 | | |
662 | 667 | | |
663 | 668 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
152 | 152 | | |
153 | 153 | | |
154 | 154 | | |
155 | | - | |
156 | | - | |
| 155 | + | |
| 156 | + | |
157 | 157 | | |
158 | 158 | | |
159 | 159 | | |
| |||
252 | 252 | | |
253 | 253 | | |
254 | 254 | | |
255 | | - | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
256 | 258 | | |
257 | 259 | | |
258 | 260 | | |
| |||
268 | 270 | | |
269 | 271 | | |
270 | 272 | | |
271 | | - | |
272 | | - | |
273 | | - | |
274 | | - | |
275 | | - | |
276 | | - | |
277 | | - | |
278 | | - | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
279 | 282 | | |
280 | 283 | | |
281 | 284 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
162 | 162 | | |
163 | 163 | | |
164 | 164 | | |
165 | | - | |
166 | | - | |
| 165 | + | |
| 166 | + | |
167 | 167 | | |
168 | 168 | | |
169 | 169 | | |
| |||
262 | 262 | | |
263 | 263 | | |
264 | 264 | | |
265 | | - | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
266 | 268 | | |
267 | 269 | | |
268 | 270 | | |
| |||
278 | 280 | | |
279 | 281 | | |
280 | 282 | | |
281 | | - | |
282 | | - | |
283 | | - | |
284 | | - | |
285 | | - | |
286 | | - | |
287 | | - | |
288 | | - | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
289 | 292 | | |
290 | 293 | | |
291 | 294 | | |
292 | 295 | | |
293 | 296 | | |
294 | 297 | | |
295 | 298 | | |
296 | | - | |
| 299 | + | |
297 | 300 | | |
298 | 301 | | |
299 | 302 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6303 | 6303 | | |
6304 | 6304 | | |
6305 | 6305 | | |
| 6306 | + | |
| 6307 | + | |
6306 | 6308 | | |
6307 | 6309 | | |
6308 | 6310 | | |
| |||
0 commit comments