You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit fe98297
Browse filesBrowse the repository at this point in the historyBrowse files
Updated the export document with custom llm documentation (pytorch#19460)
### Summary
Fixespytorch#8768
Update the custom LLM export documentation to better guide power users
adapting non-standard transformer models for ExecuTorch. The page keeps
the existing nanoGPT tutorial flow, then adds focused guidance on runner
compatibility, KV cache ownership, attention/SDPA reuse, reusable LLM
components, backend delegation considerations, and encoder-decoder model
boundaries.
This is a docs-only PR. No public APIs, runtime behavior, exporter
behavior, or backend behavior are changed.
### Test plan
```bash
lintrunner docs/source/llm/export-custom-llm.md
cd docs
make html
```
cc: @GregoryComer@nil-is-all
cc @mergennachin@AlannaBurke@larryliu0820@cccclai@helunwencser@jackzhxng@byjlw
Copy file name to clipboardExpand all lines: docs/source/llm/export-custom-llm.md
+197-2Lines changed: 197 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,9 +1,16 @@
1
1
# Exporting custom LLMs
2
2
3
-
If you have your own PyTorch model that is an LLM, this guide will show you how to manually export and lower to ExecuTorch, with many of the same optimizations as covered in the previous `export_llm` guide.
3
+
If you have your own PyTorch model that is an LLM, this guide shows how to
4
+
manually export and lower it to ExecuTorch. Use this flow when your model is not
5
+
covered by the native [`export_llm`](export-llm.md) API, is not directly handled
6
+
by [Optimum ExecuTorch](export-llm-optimum.md), or needs model-specific changes
7
+
before it can use the standard ExecuTorch LLM runtime.
4
8
5
9
This example uses Karpathy’s [nanoGPT](https://github.com/karpathy/nanoGPT), which is a minimal implementation of
6
-
GPT-2 124M. This guide is applicable to other language models, as ExecuTorch is model-invariant.
10
+
GPT-2 124M. The same manual export pattern applies broadly to PyTorch models.
11
+
However, exporting a `.pte` file and running that file with the stock LLM runners
12
+
are separate steps. To use the LLM runners, the exported model must also follow
13
+
the runtime contract described below.
7
14
8
15
9
16
## Exporting to ExecuTorch (basic)
@@ -84,6 +91,165 @@ To export, run the script with `python export_nanogpt.py` (or python3, as approp
84
91
For more information, see [Exporting to ExecuTorch](../tutorials/export-to-executorch-tutorial)<!-- @lint-ignore --> and
0 commit comments