Repository navigation
Commit 422e07c
authored
examples/llama: lower to Core ML with to_edge_transform_and_lower (pytorch#22052)
### Summary
Fixes pytorch#19634.
The Core ML branch of `_to_edge_and_lower_llama()` still used the
deprecated `export_to_edge()` + `to_backend()` split, and
`CoreMLPartitioner` logs a deprecation warning about it on every
invocation.
The split matters beyond the warning. It decomposes the graph before the
partitioner runs, so the ops Core ML has its own implementations for are
already broken into primitives by the time it sees them.
`CoreMLPartitioner.ops_to_not_decompose()` asks to keep every op Core ML
supports, and only `to_edge_transform_and_lower` honours that request.
This adds `_to_edge_and_lower_llama_coreml()`, matching the existing
xnnpack and mlx helpers, and routes to it when Core ML is the only
backend enabled.
### Scope
Enabling Core ML together with Vulkan, MPS or QNN still takes the old
combined path, which builds a single partitioner list. QNN in particular
needs the edge manager for its pass pipeline and model sharding, so it
is left alone; migrating it is a separate piece of work.
Etrecord generation comes along for free: `to_edge_transform_and_lower`
takes `generate_etrecord` directly, so the helper does not need the
deepcopy of the edge manager the old path used.
### Results
LFM2.5 350M, fp32, `max_seq_length=512`, `ios: 18`, `compute_units:
cpu_and_ne`:
| | subgraphs | delegated nodes | non-delegated | deprecation warnings |
| --- | --- | --- | --- | --- |
| before | 1 | 1156 | 13 | 1 |
| after | 1 | 893 | 13 | 0 |
Same partitioning, 263 fewer nodes for Core ML to reassemble.
`expand_copy` drops from 133 to 49 and `unsqueeze_copy` from 375 to 99.
### A note on the original report
The issue also reported a `SpecViolationError` ("Mutation node
aten_index_put_default_N is neither a buffer nor a user input") from the
split path on LFM2 hybrid models, where the short-conv
`conv_state.copy_()` decomposes to `slice_copy + index_put` and only one
of the two is recorded as the mutation source. That no longer reproduces
on `main`: the same export now completes on both the old and the new
path. Only the deprecation and the extra decomposition remain, which is
what this PR addresses.
### Test plan
Adds `examples/models/lfm2/config/lfm2_coreml_fp32.yaml` alongside the
existing `lfm2_xnnpack_fp32.yaml` and `lfm2_mlx_4w.yaml`, so the Core ML
path has a config to run:
```bash
python -m extension.llm.export.export_llm \
--config examples/models/lfm2/config/lfm2_coreml_fp32.yaml \
+base.model_class=lfm2_5_350m \
+base.params=examples/models/lfm2/config/lfm2_5_350m_config.json \
+export.max_seq_length=512 \
+export.max_context_length=512 \
+export.output_name=lfm2_coreml.pte
```1 parent 096a20a commit 422e07c
2 files changed
Lines changed: 87 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1344 | 1344 | | |
1345 | 1345 | | |
1346 | 1346 | | |
| 1347 | + | |
| 1348 | + | |
| 1349 | + | |
| 1350 | + | |
| 1351 | + | |
| 1352 | + | |
| 1353 | + | |
| 1354 | + | |
| 1355 | + | |
| 1356 | + | |
| 1357 | + | |
| 1358 | + | |
| 1359 | + | |
| 1360 | + | |
| 1361 | + | |
| 1362 | + | |
| 1363 | + | |
| 1364 | + | |
| 1365 | + | |
| 1366 | + | |
| 1367 | + | |
| 1368 | + | |
| 1369 | + | |
| 1370 | + | |
| 1371 | + | |
| 1372 | + | |
| 1373 | + | |
| 1374 | + | |
| 1375 | + | |
| 1376 | + | |
| 1377 | + | |
| 1378 | + | |
| 1379 | + | |
| 1380 | + | |
| 1381 | + | |
| 1382 | + | |
| 1383 | + | |
| 1384 | + | |
| 1385 | + | |
| 1386 | + | |
| 1387 | + | |
| 1388 | + | |
| 1389 | + | |
| 1390 | + | |
| 1391 | + | |
| 1392 | + | |
1347 | 1393 | | |
1348 | 1394 | | |
1349 | 1395 | | |
| |||
1746 | 1792 | | |
1747 | 1793 | | |
1748 | 1794 | | |
| 1795 | + | |
| 1796 | + | |
| 1797 | + | |
| 1798 | + | |
| 1799 | + | |
| 1800 | + | |
| 1801 | + | |
| 1802 | + | |
| 1803 | + | |
| 1804 | + | |
| 1805 | + | |
| 1806 | + | |
| 1807 | + | |
| 1808 | + | |
| 1809 | + | |
| 1810 | + | |
| 1811 | + | |
| 1812 | + | |
| 1813 | + | |
| 1814 | + | |
| 1815 | + | |
| 1816 | + | |
| 1817 | + | |
| 1818 | + | |
| 1819 | + | |
| 1820 | + | |
1749 | 1821 | | |
1750 | 1822 | | |
1751 | 1823 | | |
| |||
0 commit comments