Repository navigation
Commit 3546e32
authored
Arm backend: Support Ethos-U85 KV-cache export (pytorch#21913)
- Add the export/runtime plumbing needed for KV-cache SmolLM2-style
models on Ethos-U. This includes int32 token/input_pos example inputs
for Arm backends, initialized KV cache mutable buffers, calibration
dtype preservation, and an Ethos-U partitioner guard for unsupported KV
cache index ops.
- Extend the semihosting runner with input rebinding and server mode so
step-wise KV-cache decoding can reuse one FVP process across token runs.
AI-assisted-by: Codex
cc @digantdesai @freddan80 @per @zingo @oscarandersson8218 @mansnils
@Sebastian-Larsson @robell @rascani
Signed-off-by: Xingguo Li <xingguo.li@arm.com>1 parent 3ca32af commit 3546e32
4 files changed
Lines changed: 155 additions & 17 deletions
File tree
- examples
- arm/executor_runner
- models/llama
- extension/llm/export
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
193 | 193 | | |
194 | 194 | | |
195 | 195 | | |
| 196 | + | |
196 | 197 | | |
197 | 198 | | |
198 | 199 | | |
| |||
499 | 500 | | |
500 | 501 | | |
501 | 502 | | |
502 | | - | |
503 | | - | |
504 | | - | |
505 | | - | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
| 507 | + | |
| 508 | + | |
| 509 | + | |
506 | 510 | | |
507 | | - | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
508 | 516 | | |
509 | 517 | | |
510 | 518 | | |
| |||
609 | 617 | | |
610 | 618 | | |
611 | 619 | | |
| 620 | + | |
| 621 | + | |
612 | 622 | | |
613 | 623 | | |
614 | 624 | | |
| 625 | + | |
| 626 | + | |
| 627 | + | |
| 628 | + | |
| 629 | + | |
| 630 | + | |
| 631 | + | |
| 632 | + | |
| 633 | + | |
| 634 | + | |
| 635 | + | |
| 636 | + | |
| 637 | + | |
| 638 | + | |
| 639 | + | |
| 640 | + | |
| 641 | + | |
| 642 | + | |
| 643 | + | |
| 644 | + | |
| 645 | + | |
| 646 | + | |
| 647 | + | |
615 | 648 | | |
616 | 649 | | |
617 | 650 | | |
| |||
1324 | 1357 | | |
1325 | 1358 | | |
1326 | 1359 | | |
| 1360 | + | |
| 1361 | + | |
| 1362 | + | |
| 1363 | + | |
| 1364 | + | |
| 1365 | + | |
| 1366 | + | |
| 1367 | + | |
| 1368 | + | |
| 1369 | + | |
| 1370 | + | |
| 1371 | + | |
| 1372 | + | |
| 1373 | + | |
| 1374 | + | |
| 1375 | + | |
| 1376 | + | |
| 1377 | + | |
| 1378 | + | |
| 1379 | + | |
| 1380 | + | |
| 1381 | + | |
| 1382 | + | |
| 1383 | + | |
| 1384 | + | |
| 1385 | + | |
| 1386 | + | |
| 1387 | + | |
| 1388 | + | |
| 1389 | + | |
| 1390 | + | |
| 1391 | + | |
| 1392 | + | |
| 1393 | + | |
| 1394 | + | |
| 1395 | + | |
| 1396 | + | |
| 1397 | + | |
| 1398 | + | |
| 1399 | + | |
| 1400 | + | |
| 1401 | + | |
1327 | 1402 | | |
1328 | 1403 | | |
1329 | 1404 | | |
| |||
1399 | 1474 | | |
1400 | 1475 | | |
1401 | 1476 | | |
| 1477 | + | |
1402 | 1478 | | |
1403 | 1479 | | |
1404 | 1480 | | |
| |||
1414 | 1490 | | |
1415 | 1491 | | |
1416 | 1492 | | |
| 1493 | + | |
| 1494 | + | |
| 1495 | + | |
| 1496 | + | |
| 1497 | + | |
| 1498 | + | |
| 1499 | + | |
| 1500 | + | |
| 1501 | + | |
| 1502 | + | |
| 1503 | + | |
| 1504 | + | |
| 1505 | + | |
1417 | 1506 | | |
1418 | 1507 | | |
1419 | 1508 | | |
| |||
1442 | 1531 | | |
1443 | 1532 | | |
1444 | 1533 | | |
| 1534 | + | |
| 1535 | + | |
| 1536 | + | |
| 1537 | + | |
| 1538 | + | |
| 1539 | + | |
| 1540 | + | |
1445 | 1541 | | |
| 1542 | + | |
1446 | 1543 | | |
1447 | 1544 | | |
1448 | 1545 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
35 | 35 | | |
36 | 36 | | |
37 | 37 | | |
38 | | - | |
39 | 38 | | |
40 | 39 | | |
41 | 40 | | |
| |||
1263 | 1262 | | |
1264 | 1263 | | |
1265 | 1264 | | |
| 1265 | + | |
| 1266 | + | |
| 1267 | + | |
| 1268 | + | |
| 1269 | + | |
| 1270 | + | |
| 1271 | + | |
| 1272 | + | |
| 1273 | + | |
| 1274 | + | |
| 1275 | + | |
| 1276 | + | |
| 1277 | + | |
| 1278 | + | |
| 1279 | + | |
| 1280 | + | |
| 1281 | + | |
| 1282 | + | |
| 1283 | + | |
| 1284 | + | |
| 1285 | + | |
| 1286 | + | |
| 1287 | + | |
| 1288 | + | |
| 1289 | + | |
| 1290 | + | |
| 1291 | + | |
| 1292 | + | |
| 1293 | + | |
| 1294 | + | |
| 1295 | + | |
1266 | 1296 | | |
1267 | | - | |
1268 | | - | |
1269 | | - | |
1270 | | - | |
1271 | | - | |
| 1297 | + | |
| 1298 | + | |
| 1299 | + | |
1272 | 1300 | | |
1273 | 1301 | | |
1274 | 1302 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
308 | 308 | | |
309 | 309 | | |
310 | 310 | | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
311 | 317 | | |
312 | 318 | | |
313 | | - | |
314 | | - | |
| 319 | + | |
| 320 | + | |
315 | 321 | | |
316 | 322 | | |
317 | 323 | | |
318 | 324 | | |
319 | | - | |
| 325 | + | |
320 | 326 | | |
321 | 327 | | |
322 | 328 | | |
323 | | - | |
| 329 | + | |
324 | 330 | | |
325 | 331 | | |
326 | 332 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
311 | 311 | | |
312 | 312 | | |
313 | 313 | | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
314 | 321 | | |
315 | | - | |
316 | | - | |
| 322 | + | |
| 323 | + | |
317 | 324 | | |
318 | 325 | | |
319 | 326 | | |
| |||
0 commit comments